SearchManager.Search taking 90 seconds in production

Vote:
 

Hi,

When deploying to our production environment we noticed a significant performance decrease in searching for articles in commerce manager. From time to time (seems like first time, and then after 1 or so minutes) searching for articles takes 90 seconds.

After decompiling and adding stopwatches here and there, we have noticed that it is this line that takes 90 seconds.

 ISearchResults searchResults = new SearchManager(Mediachase.Commerce.Core.AppContext.Current.ApplicationName).Search((ISearchCriteria)entrySearchCriteria);

It is located in the CatalogEntrySearchDialog.cs. Any ideas on why it takes so long time in production?

//Tommy

#155151
Edited, Sep 09, 2016 14:34
Vote:
 

Hi,

Which search provider are you using? On which version?

How many rows are you getting back? If the page size is too big, then the search time can be that slow.

/Q

#155154
Sep 09, 2016 15:14
Vote:
 

Lucene search provider.

commerce version: 9.19.0

~100k products, 700k variants

But when we connect to the prod-db from our localmachine and rebuild the index locally on disc the search i fast all the time?

//Tommy

#155155
Sep 09, 2016 15:20
Vote:
 

No I meant this page size

Having it to be 1000 or All will certainly make it slow.

And 800k entries in total is on the "heavy" side for LuceneSearchProvider 

/Q

#155157
Sep 09, 2016 15:33
Vote:
 

It is set to 50.

But as i said, when we connect to the prod database locally and copy the entire index from the production machine it goes fast all the time.

In production, 90 seconds for the first search and if you do the exact same search directly after it is fast. But if we wait for ca 1 minute it is slow as hell again.

#155158
Edited, Sep 09, 2016 15:40
Vote:
 
<p>Can you run a profiler (dotTrace, prefix etc.) to see what makes it so slow?</p> <p>/Q</p>
#155160
Sep 09, 2016 15:47
Vote:
 

In prod where those index files are located? 

If it's on a file server, involvo your infrastructure team. Do a test on prod by keeping them locally. 

#155167
Sep 09, 2016 21:49
Vote:
 

Hello!

The server is a virual machine which has two hard drives and we have tried to put the lucene files on both, but get the same problem. Ie. it's not a shared network drive.

We have no problems with either of the hard drives in any other application. The site and commerce manager is hosted on one of the drives.

#155249
Sep 13, 2016 10:04
Vote:
 

As I said, I would suggest to use a profiler like dotTrace or prefix to see what is taking most of the time.

/Q

#155250
Sep 13, 2016 10:07
Vote:
 

I have profiled the application remote with dottrace now. And you can clearly see what is taking extremely long time. This is 2 searches.

https://ibin.co/2uw9ics3XLBe.png

Any tips?

#155294
Edited, Sep 13, 2016 15:21
Vote:
 
<p>My theory is that your index folder is shared between Commerce Manager and front-end site. So during the high load of front-end site, the folder is "locked" and the time you see is the time for Windows to release the lock accquired by the front-end app pool. Next time you search with same keyword, it's bit faster because the search result is cached, but it does not really help much.&nbsp;</p> <p>As I said above, 800k entries is on heavy side of Lucene (and LuceneSearchProvider). Generally I would suggest you to switch to other search providers which can handle high load better, such as Solr (via Solr35SearchProvider), or even better Find (via FindSearchProvider)&nbsp;</p> <p></p>
#155296
Sep 13, 2016 15:36
Vote:
 

But the weird thing is that it works just fine locally, with 800k entries.

Using find is not an option, since commerce manager adds a own/new index that we have to pay for. 

#155297
Sep 13, 2016 15:39
Vote:
 

When you run it locally, only your Commerce Manager AppPool accesses the folder, so no lock at all.

I've heard of issues with LuceneSearchProvider (in fact, Lucene) regarding big index/ high load site - locking is a problem. Lucene works well in small to medium index size/load, but it has its limits.

I'm just saying you have other options. It's up to you to investigate and invest. Find is a great alternative even it's not free - the investment will pay off in long term. 

#155298
Sep 13, 2016 15:48
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.