|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Hi, my search is working fine in terms of filtering and displaying results but it's incredibly slow. In CentralEventDispatcher_Action and when we get ActionName.ResultItemsFinalized I'm doing this: documentIndex = new DocumentIndex(SearchResult1.Configuration); documentIndex.Open(); sw.WriteLine(DateTime.Now.ToLongTimeString() + " Just done documentIndex.Open()"); results = new ArrayList(); foreach (ResultItem result in resultItems) { AddResultItemToResults(result, ref results); } documentIndex.Close(); sw.WriteLine(DateTime.Now.ToLongTimeString() + " Just done documentIndex.Close()"); Everything's very fast up to the open message and after the close message but between the two it's about 40 seconds for about 850 hits. Obviously I can expand on what's happening in AddResultItemToResults but it's not that much, just a simple filter on date and type of document. So first of all, does this seem incredibly slow or is it what I should be expecting? Or is there anything obvious I might be doing wrong? Many thanks. John John
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Should have said that my version was installed from Search_ASP.NET-Setup-3-1-1-UD.msi. Thanks. John John
|
|
|
|
Rank: Advanced Member
Groups: Administrators, Registered
Joined: 8/13/2004
Posts: 2,667
Location: Canada
|
Hi John, that is slow. I can't say why without seeing the code for AddResultItemToResults, since that's what's slow, right? Thanks Jim -your feedback is helpful to other users, thank you!-your feedback is helpful to other users, thank you!
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Thanks Jim. If you reckon it is too slow, I'm basically doing the right thing and nothing obvious springs to mind then I'll take another closer look at AddResultItemToResults before sending it on. Cheers. John John John
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Hi Jim, I think the bottleneck is here: Document doc = documentIndex.GetDocument(result.DocumentRecord.Uri); I need doc.LastModifiedDate so I can exclude results before a selected date AND so that I can sort what's left by date using a DocumentDateComparer. The search works five times faster if I remove this, which for my client would mean roughly 5 seconds for 800 results rather than 25 seconds. So is there a different (and much quicker) way of getting the date from the result? I've looked at the methods and there doesn't seem to be anything obvious. Many thanks. John John
|
|
|
|
Rank: Advanced Member
Groups: Administrators, Registered
Joined: 8/13/2004
Posts: 2,667
Location: Canada
|
Hi John, yes that's definitely the problem. What you need to do is write that date data to what we call the "custom data" field, which is available at search time. It's much more optimal. Doing that and removing the DocumentIndex usage will help hugely. This shows working with custom data http://keyoti.com/produc...ith%20Custom%20Data.htm
your plugin will work at index time, so you can set the custom data then to the last modified date. If you have any trouble/questions just let me know. Best Jim -your feedback is helpful to other users, thank you!-your feedback is helpful to other users, thank you!
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Hi Jim, in fact I do set the date into the custom data field (that's how I do the sorting) but what you suggest sounds massively better. to do it at index time instead. Many thanks, and I'll look forward to giving it a go today! All the best, John John
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Jim, I've got this in place now, in CentralEventDispatcher_Action: switch (e.ActionData.Name) { case ActionName.ReadText: Document doc = (Document)sender; DocumentText docText = (DocumentText)e.ActionData.Data; docText.MetaCustomData = doc.LastModifiedDate.ToShortDateString(); break; case ActionName.ResultItemsFinalized: (what I do now) ...but the handler is only called twice, for GetWordVariations and ResultItemsFinalized. ReadText is never detected. I've looked at your docs but I can't see anything to help; can you let me know what to do here please? Thanks. John John
|
|
|
|
Rank: Advanced Member
Groups: Administrators, Registered
Joined: 8/13/2004
Posts: 2,667
Location: Canada
|
John, I can't go into too much detail right now (can on Monday) but the problem is that your event handler isn't being used while indexing. Those events you're seeing (GetWordVariations and ResultItemsFinalized) are at search time. This probably means that you haven't gone the whole distance and created a plug-in dll? If you're indexing programmatically, then you don't have to make a plug-in dll, you can just attach the event handler via the configuration object, otherwise, if you're not doing it programmatically then you must create a dll and assign it in the configuration.ExternalEventHandler property. The section in the help on the "Central Event System" should explain further. Hope that helps Jim -your feedback is helpful to other users, thank you!-your feedback is helpful to other users, thank you!
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Hi Jim, many thanks for your reply yesterday. I think I now understand what to do; I've followed the instructions in the Central Event System (Plug-ins) topic in the online help. My SearchPlugin.dll works fine and I can break as described in here: public void dispatcher_Action(object sender, ActionEventArgs e) { if (e.ActionData.Name == ActionName.ReadText) { Document doc = (Document)sender; ..... } } My local document store randomly contains 98 doc/docx/pdf files and one xsl (the live one has just doc and docx). There are four problems though when I try to index using the Index Manager Tool: 1. I only get one hit on ReadText, which is the xsl file. 2. When ReadText happens, e.ActionData.Data is null. 3. There are lots of hits for IsDocumentToBeIndexed, but presumably that's no use for setting up CustomData. 4. When it finishes the Index Manager says that 0 items have been indexed, but I can do a test search perfectly well. What I'm expecting is that ReadText will happen for each document, that sender will be the Document and that e.ActionData.Data will be the DocumentText (as described in the How To Work With Custom Data topic) but this just isn't happening. Any ideas? Many thanks. John John
|
|
|
|
Rank: Advanced Member
Groups: Administrators, Registered
Joined: 8/13/2004
Posts: 2,667
Location: Canada
|
Hi John, if you are running it over an existing index, then it won't reindex those files unless they've changed. Sounds like that's the problem. Simplest thing is to delete the contents of the index dir (you can keep all .XML files to preserve your settings) and just reimport. Let me know if you prefer not to do that. Best Jim -your feedback is helpful to other users, thank you!-your feedback is helpful to other users, thank you!
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Many thanks Jim, makes perfect sense. I'll give it a go today. All the best, John John
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Hi Jim, good progress with the plugin assembly, which works perfectly when I re-import/re-index using the Index Manager Tool (CLR2). My platform is 3.5SP1. When I try to re-index with the web admin interface the CentralEventDispatcher.txt complains that it can't find the constructor in the dll. When I try to re-index with the Windows Service it complains that the dll is not in the right format. So although it works, it doesn't work (presumably because of different .NET versions?). How do I get it going with all three import/index methods? I set up from Search_ASP.NET-Setup-3-1-1-UD.msi. Thanks. John John
|
|
|
|
Rank: Advanced Member
Groups: Administrators, Registered
Joined: 8/13/2004
Posts: 2,667
Location: Canada
|
Thanks John - yes in v3 the Win Service and Web Admin app were built with our .NET 1 DLLs (in order to support all customers). In the current version we have versions for CLR 1, 2 (3, 3.5) and 4. I think your options (other than buying an upgrade) are; A. do everything with the .NET 1 DLLs (i.e. build the plugin with those references), and it seems that the Win Service is actually using the .NET 1 runtime as well, so your plug-in would need to be compiled against .NET 1 (if you don't have VS 2003, that would mean either using csc.exe manually or MSBEE). B. abandon the builtin Windows Service and Web Admin. Instead, you can find an exact copy of the Web Admin for .NET 2 inside the Search Pro demo projects, under the admin folder. It means you'd need to deploy the Web Admin yourself, rather than using our installer. If you do this you can reference the .NET 2 DLLs. As for the Win Service, I can send you our actual project source code, since it's trivial - and you can build it against .NET 2, and install it. This is pretty simple actually, even if you've never done before. Sorry it's a bit of a pickle - unfortunately supporting multiple .NET versions is a hassle. Jim -your feedback is helpful to other users, thank you!-your feedback is helpful to other users, thank you!
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Many thanks Jim, and I have the same problem up to a point with my clients so I understand. I do have 2003 in a dark corner but I'd prefer to leave it there; if you could you send me the service project that'd be great. Hopefully like other services I've done it'll just create a .msi and that'll be that. I can live without the web admin but getting the service working is all I need to crack the whole problem (hopefully!). All the best, John John
|
|
|
|
Rank: Advanced Member
Groups: Administrators, Registered
Joined: 8/13/2004
Posts: 2,667
Location: Canada
|
There isn't an MSI (it's too embedded in our build process), but here's the project http://wikisend.com/down...d/353090/winservice.zip
(you can ignore the Task class in that ZIP). You'll see a couple of batch files for installing the service (during testing), they use .NET 1.1 exes but you can edit and update the path to a newer SDK path. If you want to make an MSI, then it's just the usual case of setting the project output from this project as a custom action in the MSI project. Ah, just realised, this project is for the current version - BUT as far as I recall nothing changed in it that would affect the project (it's pretty trivial code actually) - you just need to change the references to your version DLLs. If you get stuck let me know please. Best Jim -your feedback is helpful to other users, thank you!-your feedback is helpful to other users, thank you!
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Thanks Jim; it'll be Friday before I can look at this but I'll give it a go. In case I run out of time and need to compile the plugin with 1.1 are you able to send me the required 1.1 reference assemblies, as described in the help? Add references to Keyoti.SearchEngine.Core.DLL, Keyoti.SearchEngine.License.DLL, Keyoti.Text.MSOffice.DLL, Keyoti.Text.LemmaGenerator.DLL (note that you must use the Keyoti2... versions if you use those assemblies in your toolbox - see the notes section 'References To Search DLLs'). Cheers. John John
|
|
|
|
Rank: Advanced Member
Groups: Administrators, Registered
Joined: 8/13/2004
Posts: 2,667
Location: Canada
|
Sure John - though they should be under the install dir in the .NET1 folder. Let me know if not and you want me to send (I can do it, just have to dig them out). Jim -your feedback is helpful to other users, thank you!-your feedback is helpful to other users, thank you!
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Ah, of course, they're in NETCRL_v1_BIN so that's fine. Sorry; so much of this to get used to. I'll let you know how things go on Friday. Thanks again for your instant responses. All the best, John John
|
|
|
|
Rank: Member
Groups: Registered
Joined: 9/14/2009
Posts: 27
|
Hi Jim, well, I made a 1.1 dll in the end to save time and it's now working with the Windows service in my client's live environment. Performance is fantastic! Obviously what I was doing before was technically correct but in fact completely disastrous; I was expecting quite an improvement but I'm amazed how quickly it works now. Many thanks for your help in getting this sorted; good job done! John John
|
|
|
