Title Back Colour Keyoti Title Line Title Curve
Blue Box Top

Exception caught in the import process - SearchUnit - Forum

Welcome Guest Search | Active Topics | Log In | Register

2 Pages 12>
Options
agendlin
#1 Posted : Sunday, February 23, 2014 1:30:23 AM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Hi,
I'm having this error:

Exception caught in the import process: Invalid URI: The Authority/Host could not be parsed. StackTrace: at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind)

When Service rebuild start.

I set
Service Rebuild Frequency to 1
Service Rebuild Frequency to 19

I can rebuild index manually but automatic rebuild doesn't work

Thanks,
Alex



ag
ag
Jim
#2 Posted : Sunday, February 23, 2014 2:32:40 AM
Rank: Advanced Member

Groups: Administrators, Registered

Joined: 8/13/2004
Posts: 2,669
Location: Canada
Hi Alex, can you paste the content of your indexableSources.xml file here, or email to me at support at keyoti.com please?

The errors is coming right out of Uri, and looks like there's something invalid about it - I don't know why you'd see different behavior with the Service though, they both use the same code base....

Regards
Jim

-your feedback is helpful to other users, thank you!

-your feedback is helpful to other users, thank you!


agendlin
#3 Posted : Sunday, February 23, 2014 8:18:43 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Hi Jim,

Here is:


<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<DataSources>
<LastUsedID>5</LastUsedID>
<DataSource ID="1" type="Website" location="http:/localhost:8081" query="" uniqueColumn="" resultURL="" ExtensionData="@" />
<DataSource ID="2" type="Website" location="http:/localhost:" query="" uniqueColumn="" resultURL="" ExtensionData="@" />
<DataSource ID="3" type="FileSystemDocumentStore" location="c:\WebData\" query="/" uniqueColumn="" resultURL="" ExtensionData="@@True@" />
<DataSource ID="4" type="FileSystemDocumentStore" location="c:\WebData\" query="http://localhost:8081/WebData/" uniqueColumn="" resultURL="" ExtensionData="@@True@" />
<DataSource ID="5" type="Website" location="http://localhost:8081/" query="" uniqueColumn="" resultURL="" ExtensionData="@" />
</DataSources>

Thanks,
Alex

ag
ag
agendlin
#4 Posted : Sunday, February 23, 2014 8:26:38 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Hi Jim,
I think I know what is going on. I have bad entries in indexableSources.xml. I delete them, now I just have to wait for process to start. Let you know if this going to work.

Thanks,
Alex

ag
ag
agendlin
#5 Posted : Sunday, February 23, 2014 10:12:31 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Thanks for pointing me in the right direction. It does work now. Any way I have another question. How can I trigger re-indexing any time I add new file or new html page?

Thanks,
Alex

ag
ag
Jim
#6 Posted : Monday, February 24, 2014 3:19:37 AM
Rank: Advanced Member

Groups: Administrators, Registered

Joined: 8/13/2004
Posts: 2,669
Location: Canada
I wouldn't suggest reindexing everything everytime you add a document, so this is the best approach

http://keyoti.com/produc...porting.htm#incremental

Jim

-your feedback is helpful to other users, thank you!

-your feedback is helpful to other users, thank you!


agendlin
#7 Posted : Monday, February 24, 2014 8:41:08 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Hi Jim,
Can you tell me how to implement this kind of search below:
I was be able to find examples and implement location and content search. But I need little bit more complex, like in this example:


Search For:
Match exact phrase Sort By: Last Modified Descending
Search for all Words Title Ascending
Match any of the words Search Rank


Thank You,
Alex
ag
agendlin
#8 Posted : Monday, February 24, 2014 9:23:31 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
I re-post the example of what I need

Search For:
Match exact phrase
Search for all Words
Match any of the words

Sort By:
Last Modified
Title
Match any of the words

Sort By:
Ascending
Descending


Thanks,
Alex


ag
ag
Jim
#9 Posted : Tuesday, February 25, 2014 3:06:37 AM
Rank: Advanced Member

Groups: Administrators, Registered

Joined: 8/13/2004
Posts: 2,669
Location: Canada
Alex,

quote:

Search For:
Match exact phrase
Search for all Words
Match any of the words



This is implemented by the query syntax (as in google). Exact phrase just means wrapping in quotes.
Search for all words is the default (AND operator)
Match any of the words means OR operator, eg. word1 OR word2

If you want to make it so that any word is matched, without using "OR" explicitly, then you can do this in the search page codebehind (Page_Load)

SearchResult1.Configuration.ImpliedLogicOperator = Keyoti.SearchEngine.Search.ImpliedLogicOperator.Or;

Likewise set to .And to set it back to 'match all words'.

Similarly, to wrap the query with quotes, in codebehind

SearchResult1.QueryExpression = "\"" + SearchResult1.QueryExpression + "\"";

(maybe check there aren't already quotes first).


Sorting:
Here's an article on sorting by a date
http://keyoti.com/produc...ith%20Custom%20Data.htm

You can modify it to order by Title. Am unsure what you mean by 'Match any of the words'? (we already rank by frequency of matches).
Last modified; you would need to store that date in the CustomData (as described in the article). You can get 'last modified' from the server response, in the plugin code.

Handle the ResponseFromServerReceived event, eg
Code:

Uri lastResponseURI;
DateTime lastResponseLastModified;

private void CentralEventDispatcher_Action(object sender, Keyoti.SearchEngine.Events.ActionEventArgs e)

{

    if(e.ActionData.Name== ActionName.ResponseFromServerReceived )

    {

       lastResponseURI = (e.ActionData.Data As HttpWebResponse).ResponseUri
       lastResponseLastModified = (e.ActionData.Data As HttpWebResponse).LastModified;


    }

if(e.ActionData.Name== ActionName.ReadText)

    {

        if(lastResponseURI  ==  (sender as Document).Uri){

        (e.ActionData.Data as DocumentText).MetaCustomData = lastResponseLastModified.ToShortDateString();
}

    }

}


Best
Jim

-your feedback is helpful to other users, thank you!

-your feedback is helpful to other users, thank you!


agendlin
#10 Posted : Wednesday, February 26, 2014 3:50:20 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Hi Jim,
I Create plug-in to handle ResponseFromServerReceived event, using the your code. Its build and initialized OK when I set Plug-in Path, but its not doing anything. I was trying to debug and it would not attached to the process
with an arror: The breakpoint will not currently be hit. No symbols have been loaded for this document
Please let me know what I doing wrong.

Thanks
Alex

Here is my plug- code:

Imports System
Imports System.Collections.Generic
Imports System.Text
Imports Keyoti.SearchEngine.Events
Imports Keyoti.SearchEngine
Imports Keyoti.SearchEngine.Documents
Imports Keyoti.SearchEngine.DataAccess
Imports Keyoti.SearchEngine.DataAccess.IndexableSourceRecords

Public Class ExternalEventHandler

Private dispatcher As IEventDispatcher

Private conf As Configuration

Public lastResponseURI As Uri

Public lastResponseLastModified As DateTime

''' <summary>
''' New, attaches event handlers.
''' </summary>
''' <param name="dispatcher">The object which fires events that this plug-in handles.</param>
''' <param name="conf">The engine configuration.</param>
Public Sub New(ByVal dispatcher As IEventDispatcher, ByVal conf As Configuration)
MyBase.New()
DataAccess.Log.WriteLogEntry("Plug-in Template Project", "Initialized", conf)
'AddHandler dispatcher.Action, AddressOf Me.dispatcher_Action
AddHandler dispatcher.Action, AddressOf Me.CentralEventDispatcher_Action
AddHandler dispatcher.NeedObject, AddressOf Me.dispatcher_NeedObject
Me.dispatcher = dispatcher
Me.conf = conf
End Sub

''' <summary>
''' Removes the handlers, so that the object can be disposed of.
''' </summary>
''' <remarks>This is called when the DLL is unloaded.</remarks>
Public Sub DetachHandlers()
If (Not (dispatcher) Is Nothing) Then
'RemoveHandler dispatcher.Action, AddressOf Me.dispatcher_Action
RemoveHandler dispatcher.Action, AddressOf Me.CentralEventDispatcher_Action
RemoveHandler dispatcher.NeedObject, AddressOf Me.dispatcher_NeedObject
End If
End Sub

''' <summary>
''' Handles ACTION events. This method is called as the engine performs various actions.
''' </summary>
Public Sub dispatcher_Action(ByVal sender As Object, ByVal e As ActionEventArgs)
'Log everything - comment this line after debugging to optimize speed.
DataAccess.Log.WriteLogEntry("Plug-in Template Project", e.ActionData.Name.ToString, conf)

Dim document As Document
Dim documentRecord As DocumentRecord
Dim word As Word

Try


Select Case (e.ActionData.Name)
Case ActionName.ReadingText
Dim documentText As DocumentText = CType(e.ActionData.Data, DocumentText)
documentText.AppendText("append some test words zzz", conf)
documentText.PrependText("prepend some test words aaa", conf)
Case ActionName.AutoCompleteSuggestionsGenerated
Dim suggestionPairs As ArrayList = CType(e.ActionData.Data, ArrayList)
Dim helper As Keyoti.SearchEngine.Web.SearchBoxAutoCompleteHelper = CType(sender, Keyoti.SearchEngine.Web.SearchBoxAutoCompleteHelper)
Dim textEnteredByUser As String = helper.Query
Dim wordsInText() As String = helper.Segments
suggestionPairs.Add(New Keyoti.SearchEngine.Web.SearchBoxAutoCompleteHelper.SuggestionResultPair("testQuery", 99))
Case ActionName.DocumentBeingCrawled
document = CType(CType(e.ActionData.Data, Object())(0), Document)
Dim links As System.Collections.ArrayList = CType(CType(e.ActionData.Data, Object())(1), System.Collections.ArrayList)
Case ActionName.AutoAssignContent
documentRecord = CType(e.ActionData.Data, DocumentRecord)
Case ActionName.AutoAssignLocation
documentRecord = CType(e.ActionData.Data, DocumentRecord)
Case ActionName.AutoAssignSecurityGroup
documentRecord = CType(e.ActionData.Data, DocumentRecord)
Case ActionName.CalculateWordRelevancies
Dim words As WordCollection = CType(e.ActionData.Data, WordCollection)
document = CType(sender, Document)
If document.Uri.AbsolutePath.Contains("/important/") Then
For Each word2 As Word In words
word2.Weight = (word2.Weight * 2)
'boost the weight of all words by 2, if they're in the /important/ subdir.
Next
End If
Case ActionName.GetWordVariations
word = CType(CType(e.ActionData.Data, Object())(0), Word)
Dim variations As System.Collections.ArrayList = CType(CType(e.ActionData.Data, Object())(1), System.Collections.ArrayList)
If (word.WordContent = "test") Then
variations.Add("testword")
End If
'make the engine look for 'testword' as well, whenever 'test' is searched for.
Case ActionName.ImportStarted
Dim sourceRecord As IndexableSourceRecord = CType(e.ActionData.Data, IndexableSourceRecord)
Case ActionName.ImportFinished
If (TypeOf e.ActionData.Data Is Exception) Then
'something went wrong with the import
Dim exception As String = CType(e.ActionData.Data, Exception).Message
Else
'import was successful
Dim uris As ArrayList = CType(e.ActionData.Data, ArrayList)
For Each importedItem As Uri In uris

Next
End If
Case ActionName.IsDocumentToBeCrawled
Dim data As IsDocumentToBeCrawledEventData = CType(e.ActionData.Data, IsDocumentToBeCrawledEventData)
Dim doc As Document = data.Document
'use some logic to determine if 'data.WillCrawl' should be set to false, which will prevent the document from being crawled.
Case ActionName.IsDocumentToBeIndexed
Dim indexData As IsDocumentToBeIndexedEventData = CType(e.ActionData.Data, IsDocumentToBeIndexedEventData)
document = indexData.Document
'Use some logic to determine if 'indexData.WillIndex' should be set to false, which will prevent the document from being indexed.
Case ActionName.RequestingUri
Dim request As System.Net.HttpWebRequest = CType(e.ActionData.Data, System.Net.HttpWebRequest)
'Change any properties in request as needed.
Case ActionName.ResponseFromServerReceived
Dim response As System.Net.HttpWebResponse = CType(e.ActionData.Data, System.Net.HttpWebResponse)
'Process as needed
Case ActionName.ResultPreviewTextLoaded
Dim documentPreviewText As StringBuilder = CType(e.ActionData.Data, StringBuilder)
Dim uri As String = CType(sender, Keyoti.SearchEngine.Web.SearchResultPreviewHelper).Uri
'Change the document text, to put the Uri at the start of the text.
'Remember that if you reassign documentPreviewText to a new object, it will not be used by the previewer.
documentPreviewText.Insert(0, ("Document URL::" _
+ (uri + "" & vbCrLf & vbCrLf)))
End Select


Catch ex As Exception
DataAccess.Log.WriteLogEntry("Plug-in Template Project", "Exception occurred: " & ex.ToString, conf)
End Try


End Sub

''' <summary>
''' Handles need object events. This method is called as the engine requires new instances of various classes.
''' </summary>
Public Sub dispatcher_NeedObject(ByVal sender As Object, ByVal e As NeedObjectEventArgs)
'leave blank unless needed, to use default objects.
End Sub

Public Sub CentralEventDispatcher_Action(ByVal sender As Object, ByVal e As ActionEventArgs)

Try

If e.ActionData.Name = ActionName.ResponseFromServerReceived Then

lastResponseURI = CType(e.ActionData.Data, Net.HttpWebResponse).ResponseUri
lastResponseLastModified = (CType(e.ActionData.Data, Net.HttpWebResponse)).LastModified

End If

If e.ActionData.Name = ActionName.ReadText Then

If lastResponseURI = CType(sender, Document).Uri Then
CType(e.ActionData.Data, DocumentText).MetaCustomData = lastResponseLastModified.ToShortDateString()
End If

End If

Catch ex As Exception

End Try

End Sub

End Class



ag
ag
Jim
#11 Posted : Wednesday, February 26, 2014 5:07:45 PM
Rank: Advanced Member

Groups: Administrators, Registered

Joined: 8/13/2004
Posts: 2,669
Location: Canada
The code looks OK, did you try to attach to the index manager?

You can also add lines like this

DataAccess.Log.WriteLogEntry("PluginProject", "Ready", conf)

to CentralEventDispatcher_Action, and provided logging is enabled in your configuration, you should get a log file called PluginProject.txt

Jim


-your feedback is helpful to other users, thank you!

-your feedback is helpful to other users, thank you!


agendlin
#12 Posted : Wednesday, February 26, 2014 5:34:34 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Yes, I did and It did not work,
Try to create PluginProject.txt, not working. It did not create log file

But the CentralEventDispatcher.txt output is Success:

02/26/2014 11:20 Success, initialized external event handler assembly @ Plugins\Search-Plugin-VB\bin\Debug\VB-PluginTemplate.dll

I don't think the plug-in is working, otherwise it would create the log file.

Please let me know if you have any ideas.

Thanks,
Alex

ag
agendlin
#13 Posted : Wednesday, February 26, 2014 7:33:34 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
OK. I did figure it out. I finally was able to debug it. The plug-in was working, but I had to modify CentralEventDispatcher_Action. So Now I've got the date. I will try to sort it.

Thanks,
Alex

ag
ag
agendlin
#14 Posted : Wednesday, February 26, 2014 8:19:30 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Hi Jim,
I've got one more question. So now I have plug-in with centralEventDispatcher, and from page_load I call the same event to trigger the date sort.

SearchResult1.FilterLoadLevel = Keyoti.SearchEngine.Search.FilterLoadLevel.Everything
AddHandler SearchResult1.Configuration.CentralEventDispatcher.Action, AddressOf Dispatcher_Action

So instead of get to Dispatcher_Action function which part of the same class, its going to plug-in.

How I can call the Dispatcher_Action now or I have to move it to plug-in ?

thanks,
Alex

ag
ag
Jim
#15 Posted : Wednesday, February 26, 2014 9:16:41 PM
Rank: Advanced Member

Groups: Administrators, Registered

Joined: 8/13/2004
Posts: 2,669
Location: Canada
Alex, it should call both the plugin and your Dispatcher_Action method in codebehind. If it isn't (for some reason, that I can't guess), then it is safe to put the code in the plug-in.

Jim

-your feedback is helpful to other users, thank you!

-your feedback is helpful to other users, thank you!


agendlin
#16 Posted : Thursday, February 27, 2014 4:45:28 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Hi Jim,
Finally got the sort work. Apparently the CentralEventDispatcher_Action start work after the result is on the page, and I was trying to set sort as an option.

Thanks for your help,

Alex
ag
agendlin
#17 Posted : Thursday, February 27, 2014 7:14:40 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Hi Jim I've got one more for you.
I put this logic in code behind to update the search with this options All Words, Any Word, Exact expression
Can you look at the code below if this is correct. It doesn't do anything.

Select Case CType(Me.cboQueryType.SelectedValue, String)
Case "all"
SearchResult1.Configuration.ImpliedLogicOperator = Keyoti.SearchEngine.Search.ImpliedLogicOperator.And
Case "any"
SearchResult1.Configuration.ImpliedLogicOperator = Keyoti.SearchEngine.Search.ImpliedLogicOperator.Or
Case "exact"
SearchResult1.QueryExpression = """" & SearchResult1.QueryExpression & """"
End Select


Thanks,
Alex
ag
agendlin
#18 Posted : Thursday, February 27, 2014 9:50:21 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Hi Jim,
If I just set SearchResult1.Configuration.ImpliedLogicOperator in page load with out select case its working, but I need to make a selection. like you provide in forum search. Can you please give me a hint how to do that.

Thanks,
Alex

ag
ag
Jim
#19 Posted : Thursday, February 27, 2014 9:55:20 PM
Rank: Advanced Member

Groups: Administrators, Registered

Joined: 8/13/2004
Posts: 2,669
Location: Canada
Hi Alex, have you used your debugger to see what is coming out of this

CType(Me.cboQueryType.SelectedValue, String)

Does it fit with your cases?

Jim

-your feedback is helpful to other users, thank you!

-your feedback is helpful to other users, thank you!


agendlin
#20 Posted : Thursday, February 27, 2014 10:05:42 PM
Rank: Member

Groups: Registered

Joined: 2/23/2014
Posts: 24
Yes the Select statement is working fine, I debug through it and it get to the right option and execute the configuration statement, but nothing happened. This is suppose to be very simple.

Alex

ag
ag
2 Pages 12>
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.




About | Contact | Site Map | Privacy Policy

Copyright © 2002- Keyoti Inc.