|
SORTED view in XPages after a fulltext search
Julian Buss, January 13th, 2010 11:53:42
Tags: XPages
Everyone knows Nathan T. Freemans Tigerstyle demo. It's a good demo, and it shows how to overcome the limitation that a filtered view (filtered by a category, for example) is not sorted anymore in the XPages world.
But there is one usecase left: if you did a fulltext search on a view, then the results are sorted "by relevance", whatever that means. In the user's eye the view is not sorted at all after the search. So, if you thought you could build cool "filterable" views by building some nice UI in that a user can select some criterias, compute a fulltext query out of that, do a fulltext search and display the result - forget it. The fact that the results are not sorted is a no-go for most usecases. And, beside that, the user also expects that he can re-sort the view after the search by any column. Bad, isn't it? IBM is aware of that problem, and hopefully they will come up with a build-in solution in the near future. But that's not for sure. But, you know me, I would not write such a blog entry if I would not have any solution on the horizon :-) Indeed, I spent the best part of yesterday's evening on that problem, without luck. Even today under the shower only a faint of an idea crossed my mind (normally when cracking that sort of problems a shower in the morning always helps...). Luckily, the faint got stronger and envolved to a real idea, for which I did some prototyping and it works so far. So, my solution / workaround (whatever you want to call it) needs some testing regarding memory consumption et al, but it works basically and I think we can use it in production in the near future. I hope I can post code when my tests are finished, but for you other XPages pros out there, here is an outline of the idea: - get the NotesView - do a ftsearch on the NotesView - get the NotesViewEntryCollection of the already searched view - loop over all NotesViewEntry elements of the collection (which has max 5000 entries) - add each NotesViewEntry to a java.util.TreeMap with an entry's column value as the key - return the values of the TreeMap to a repeat control. Quite easy, and just some lines of code. Works together with a pager control and is relatively fast. On a slow server at my office the whole process takes under two seconds for a result of about 4.000 NotesViewEntrys. The beauty in that solution is that you don't need to sort anymore, since the TreeMap is sorted automatically. What do you think about that solution? It there anything left to optimize?
Comments (11) | Permanent Link
1)
SORTED view in XPages after a fulltext search
I don't think "everyone" knows the Tigerstyle demo, but thanks for the tip of the hat. Your idea will work, but you should bear in mind that the ViewEntries will be recycled between pages. So you're going to have to come up with a wrapper that can handle request-safe retrieval of the underlying Notes object. For the record, Tigerstyle does do full-text searches. The search term box isn't showing up right now for some reason I don't have time to troubleshoot, but the code is all in there and I've done about 100 different searches in testing. Works like a charm. It's pretty cool to to an FT search, AND apply a column filter, and still resort the results by yet-another column. 2)
SORTED view in XPages after a fulltext search
the recycling is not a problem yet... what I'm still needs to nail down is if the HTTP server frees up memory automatically, or if I need to recycle the notesViewEntry by myself like in pure Java. In my earlier tests with other algorithms the HTTP eat up lots of memory, and didn't free it after the request was completed. 3)
SORTED view in XPages after a fulltext search
I posted one method for sorting the results of a FT search and displaying it in a repeat control of an XPage. I don't claim that it's the best or most efficient method but it worked for my purposes. { Link } 4)
SORTED view in XPages after a fulltext search
Ernie, thanks for the link! Yes, that's a possible solution, too. I tested a similar approach, but in my case it took too long to compute and needed too much memory. 5)
SORTED view in XPages after a fulltext search
Not sure if you follow my blog..? If not, here's my best attempt (so far) on the issue of sortable search results. { Link } The search interface itself in the demo is crappy, so please ignore that, and take a look at the (messy) code that powers the Data Table (if you're interested). I'm really happy with the sorting/pagination performance. 6)
SORTED view in XPages after a fulltext search
Tommy, yes, I follow your blog and I saw your post... I already made a comment to it, but for whatever reason the comment is lost. I think your idea for sorting is a good one, too! Nevertheless does IBM have to come up with a general solution so that we can spare the workarounds :-) 7)
SORTED view in XPages after a fulltext search
I've opened up the comment system now, so hopefully I'll get more input on my weird experiments. :) 8)
SORTED view in XPages after a fulltext search
Hi Julian, thank you for sharing your solution with us. But for the Beginner it is very difficult to understand. maybe you could put your solution in a small sample database. That would be a great Help. Thanks Sezgin 9)
SORTED view in XPages after a fulltext search
Hi Sezgin, sorry, I don't have time to provide a sample db in near future. 10)
SORTED view in XPages after a fulltext search
hwta became the code for this: - get the NotesView - do a ftsearch on the NotesView - get the NotesViewEntryCollection of the already searched view - loop over all NotesViewEntry elements of the collection (which has max 5000 entries) - add each NotesViewEntry to a java.util.TreeMap with an entry's column value as the key - return the values of the TreeMap to a repeat control. 11)
SORTED view in XPages after a fulltext search
Hi Julian, we had the same problem. But in our case the views are not too large (< 1000) and so I decided to do the sorting on the client via JavaScript (paging must be disabled) I found a scriptlib that I adopted a little, and I think it works very well (at a table with mutltiple columns and img sorting is very fast). You just have to add a class="sortable" Here is the code: function initSortableView(viewPanelId) { //alert("viewPanelId=" + viewPanelId); sortables_init(); } /* Table sorting script by Joost de Valk, check it out at { Link } Based on a script from { Link } Distributed under the MIT license: { Link } . Copyright (c) 1997-2007 Stuart Langridge, Joost de Valk. Version 1.5.7 */ /* You can change these values */ var image_path = "/domjava/xsp/theme/common/images/"; var image_up = "sort_both_ascending.gif"; var image_down = "sort_both_descending.gif"; var image_none = "sort_none.gif"; var europeandate = true; var alternate_row_colors = true; /* Don't change anything below this unless you know what you're doing */ // addEvent(window, "load", sortables_init); var SORT_COLUMN_INDEX; var thead = false; function sortables_init() { // Find all tables with class sortable and make them sortable if (!document.getElementsByTagName) return; tbls = document.getElementsByTagName("table"); for (ti=0;ti<tbls.length;ti++) { thisTbl = tbls[ti]; if (((' '+thisTbl.className+' ').indexOf("sortable") != -1) && (thisTbl.id)) { ts_makeSortable(thisTbl); } } } function ts_makeSortable(t) { if (t.rows && t.rows.length > 0) { if (t.tHead && t.tHead.rows.length > 0) { var firstRow = t.tHead.rows[t.tHead.rows.length-1]; thead = true; } else { var firstRow = t.rows[0]; } } if (!firstRow) return; // We have a first row: assume it's the header, and make its contents clickable links for (var i=0;i<firstRow.cells.length;i++) { var cell = firstRow.cells; var txt = ts_getInnerText(cell); //alert(cell.innerHTML); if ( (cell.className != "unsortable") && (cell.className.indexOf("unsortable") == -1) && (cell.innerHTML.indexOf("unsortable") == -1) ) { cell.innerHTML = '<a href="#" class="sortheader xspPanelViewColumnHeader" onclick="ts_resortTable(this, '+i+');return false;">'+txt+'<span class="sortarrow"> <img src="' + image_path + image_none + '" alt="↓"/></span></a>'; } } if (alternate_row_colors) { alternate(t); } } function ts_getInnerText(el) { if (typeof el == "string") return el; if (typeof el == "undefined") { return el }; if (el.innerText) return el.innerText; //Not needed but it is faster var str = ""; var cs = el.childNodes; var l = cs.length; for (var i = 0; i < l; i++) { switch (cs.nodeType) { case 1: //ELEMENT_NODE str += ts_getInnerText(cs); break; case 3: //TEXT_NODE str += cs.nodeValue; break; } } return str; } function ts_resortTable(lnk, clid) { var span; for (var ci=0;ci<lnk.childNodes.length;ci++) { if (lnk.childNodes[ci].tagName && lnk.childNodes[ci].tagName.toLowerCase() == 'span') span = lnk.childNodes[ci]; } var spantext = ts_getInnerText(span); var td = lnk.parentNode; var column = clid || td.cellIndex; var t = getParent(td,'TABLE'); // Work out a type for the column if (t.rows.length <= 1) return; var itm = ""; var i = 0; while (itm == "" && i < t.tBodies[0].rows.length) { var itm = ts_getInnerText(t.tBodies[0].rows.cells[column]); itm = trim(itm); if (itm.substr(0,4) == "<!--" || itm.length == 0) { itm = ""; } i++; } if (itm == "") return; sortfn = ts_sort_caseinsensitive; if (itm.match(/^\d\d[\/\.-][a-zA-z][a-zA-Z][a-zA-Z][\/\.-]\d\d\d\d$/)) sortfn = ts_sort_date; if (itm.match(/^\d\d[\/\.-]\d\d[\/\.-]\d\d\d{2}?$/)) sortfn = ts_sort_date; if (itm.match(/^-?[£$€Û¢´]\d/)) sortfn = ts_sort_numeric; if (itm.match(/^-?(\d+[,\.]?)+(E[-+][\d]+)?%?$/)) sortfn = ts_sort_numeric; SORT_COLUMN_INDEX = column; var firstRow = new Array(); var newRows = new Array(); for (k=0;k<t.tBodies.length;k++) { for (i=0;i<t.tBodies[k].rows[0].length;i++) { firstRow = t.tBodies[k].rows[0]; } } for (k=0;k<t.tBodies.length;k++) { if (!thead) { // Skip the first row for (j=1;j<t.tBodies[k].rows.length;j++) { newRows[j-1] = t.tBodies[k].rows[j]; } } else { // Do NOT skip the first row for (j=0;j<t.tBodies[k].rows.length;j++) { newRows[j] = t.tBodies[k].rows[j]; } } } newRows.sort(sortfn); if (span.getAttribute("sortdir") == 'down') { ARROW = ' <img src="'+ image_path + image_down + '" alt="↓"/>'; newRows.reverse(); span.setAttribute('sortdir','up'); } else { ARROW = ' <img src="'+ image_path + image_up + '" alt="↑"/>'; span.setAttribute('sortdir','down'); } // We appendChild rows that already exist to the tbody, so it moves them rather than creating new ones // don't do sortbottom rows for (i=0; i<newRows.length; i++) { if (!newRows.className || (newRows.className && (newRows.className.indexOf('sortbottom') == -1))) { t.tBodies[0].appendChild(newRows); } } // do sortbottom rows only for (i=0; i<newRows.length; i++) { if (newRows.className && (newRows.className.indexOf('sortbottom') != -1)) t.tBodies[0].appendChild(newRows); } // Delete any other arrows there may be showing var allspans = document.getElementsByTagName("span"); for (var ci=0;ci<allspans.length;ci++) { if (allspans[ci].className == 'sortarrow') { if (getParent(allspans[ci],"table") == getParent(lnk,"table")) { // in the same table as us? allspans[ci].innerHTML = ' <img src="'+ image_path + image_none + '" alt="↓"/>'; } } } span.innerHTML = ARROW; alternate(t); } function getParent(el, pTagName) { if (el == null) { return null; } else if (el.nodeType == 1 && el.tagName.toLowerCase() == pTagName.toLowerCase()) { return el; } else { return getParent(el.parentNode, pTagName); } } function sort_date(date) { // y2k notes: two digit years less than 50 are treated as 20XX, greater than 50 are treated as 19XX dt = "00000000"; if (date.length == 11) { mtstr = date.substr(3,3); mtstr = mtstr.toLowerCase(); switch(mtstr) { case "jan": var mt = "01"; break; case "feb": var mt = "02"; break; case "mar": var mt = "03"; break; case "apr": var mt = "04"; break; case "may": var mt = "05"; break; case "jun": var mt = "06"; break; case "jul": var mt = "07"; break; case "aug": var mt = "08"; break; case "sep": var mt = "09"; break; case "oct": var mt = "10"; break; case "nov": var mt = "11"; break; case "dec": var mt = "12"; break; // default: var mt = "00"; } dt = date.substr(7,4)+mt+date.substr(0,2); return dt; } else if (date.length == 10) { if (europeandate == false) { dt = date.substr(6,4)+date.substr(0,2)+date.substr(3,2); return dt; } else { dt = date.substr(6,4)+date.substr(3,2)+date.substr(0,2); return dt; } } else if (date.length == 8) { yr = date.substr(6,2); if (parseInt(yr) < 50) { yr = '20'+yr; } else { yr = '19'+yr; } if (europeandate == true) { dt = yr+date.substr(3,2)+date.substr(0,2); return dt; } else { dt = yr+date.substr(0,2)+date.substr(3,2); return dt; } } return dt; } function ts_sort_date(a,b) { dt1 = sort_date(ts_getInnerText(a.cells[SORT_COLUMN_INDEX])); dt2 = sort_date(ts_getInnerText(b.cells[SORT_COLUMN_INDEX])); if (dt1==dt2) { return 0; } if (dt1<dt2) { return -1; } return 1; } function ts_sort_numeric(a,b) { var aa = ts_getInnerText(a.cells[SORT_COLUMN_INDEX]); aa = clean_num(aa); var bb = ts_getInnerText(b.cells[SORT_COLUMN_INDEX]); bb = clean_num(bb); return compare_numeric(aa,bb); } function compare_numeric(a,b) { var a = parseFloat(a); a = (isNaN(a) ? 0 : a); var b = parseFloat(b); b = (isNaN(b) ? 0 : b); return a - b; } function ts_sort_caseinsensitive(a,b) { aa = ts_getInnerText(a.cells[SORT_COLUMN_INDEX]).toLowerCase(); bb = ts_getInnerText(b.cells[SORT_COLUMN_INDEX]).toLowerCase(); if (aa==bb) { return 0; } if (aa<bb) { return -1; } return 1; } function ts_sort_default(a,b) { aa = ts_getInnerText(a.cells[SORT_COLUMN_INDEX]); bb = ts_getInnerText(b.cells[SORT_COLUMN_INDEX]); if (aa==bb) { return 0; } if (aa<bb) { return -1; } return 1; } function addEvent(elm, evType, fn, useCapture) // addEvent and removeEvent // cross-browser event handling for IE5+, NS6 and Mozilla // By Scott Andrew { if (elm.addEventListener){ elm.addEventListener(evType, fn, useCapture); return true; } else if (elm.attachEvent){ var r = elm.attachEvent("on"+evType, fn); return r; } else { alert("Handler could not be removed"); } } function clean_num(str) { str = str.replace(new RegExp(/[^-?0-9.]/g),""); return str; } function trim(s) { return s.replace(/^\s+|\s+$/g, ""); } function alternate(table) { // Take object table and get all it's tbodies. var tableBodies = table.getElementsByTagName("tbody"); // Loop through these tbodies for (var i = 0; i < tableBodies.length; i++) { // Take the tbody, and get all it's rows var tableRows = tableBodies.getElementsByTagName("tr"); // Loop through these rows // Start at 1 because we want to leave the heading row untouched for (var j = 0; j < tableRows.length; j++) { // Check if j is even, and apply classes for both possible results if ( (j % 2) == 0 ) { if ( !(tableRows[j].className.indexOf('odd') == -1) ) { tableRows[j].className = tableRows[j].className.replace('odd', 'even'); } else { if ( tableRows[j].className.indexOf('even') == -1 ) { tableRows[j].className += " even"; } } } else { if ( !(tableRows[j].className.indexOf('even') == -1) ) { tableRows[j].className = tableRows[j].className.replace('even', 'odd'); } else { if ( tableRows[j].className.indexOf('odd') == -1 ) { tableRows[j].className += " odd"; } } } } } } Kind regards David |
|

