What's the maximum number of products?

[fleetwood]

We have around 13,000 products with pictures, with an average of 10 products per page.
Uploads take around 15-20 mins, although a full refresh can take 2hours plus.
The biggest time is taken with the checking procedures, rather than uploading.
I find there is a larger strain on the orders tables, than on the product tables.
If we don't clear down old orders every 6 months or so, the crashes start to get more regular.
Regular compacting of the db is essential, to keep the size manageable - Access is terrible for wasting space, especially as the size of the DB grows.

I would suggest that 50,000 may be pushing your luck.

[los_design]

20,000 products maximum - depends on layout

Hi Guys

Just tried (without success) to get a db of 19,750ish products up onto los servers. The csv file is perfecto and imports sweet, but jeez louise, when you begin to upload....it is a no go!

I can get 50k live no problem as long as I have 40-50 products per page, listed, no image, no description.

When you switch to product per page, in my experience the maximum is around the 5k mark before actinic wobbles.

Hope this helps the original post and to concur with other comments, 50k needs to be SQL/MySQL or .net based.

[fleetwood]

Darren

Out of interest, what was the size in KB of ActinicCatalog.dbf for the 20,000 product failure?

Martin

[los_design]

Hi Martin

Long time no speak....anywho, to answer your question, it was 49.7mb and I have trimmed it down to 37.9 and tried again but failed so off to trim again...

Putting it into MySQL tomorrow for a comparison as I need to be able to upload many, many products per page and me thinks me and actinic may well part company for this particular project,

You can see the testing on los, currently holding just under 5k prod per page without issue.

[fleetwood]

Hi Darren

megacity currently weighs in at 80MB plus when compacted, and uploads fine, although I am looking at the prospect of splitting into two stores, as I would like to change a lot of the products to one per section, which will eat up any available space I have.

I'm still with Act 5 (despite purchasing 7 & 8 - I just love giving Actinic my money!), so maybe that has something to do with how large db uploads are handled!

If your MySQL test is with Act Enterprise, I'd be interested in your findings, although I do hate those 'enquire for pricing' products, as that often means 'expensive'.

[los_design]

Hi Martin

In short no, my testing is being phased out on Actinic and is to be based on a hybrid of an in-house built xml/mysql app bits and peices (with permission) bastardised from an american associate who has built a superb cart specifically for dw.

Will keep you posted but maybe best if you drop me a line and I will update you off forum as it will not be actinic based.

Meanwhile back on planet Actinic....80mb. So why the chuff is it struggling with mine?

The two latest imports failed on two sites it takes 1.5 hours to check etc then gets halfway through the refresh then just stops and goes back to static.

I have it set as product per page, the only script added is a little adsense magic dust but nothing else.

[fleetwood]

That 80MB is the total db, which includes around 6 months worth of orders.

This is well below maximum Access db size.

If I understand correctly, you have been testing with a new site, presumably without any orders data yet. Its possible that my products tables are smaller than yours, but the overall database is larger when including the orders tables.

I'm guessing that the problem may be related to table/index size, rather than overall db size.

We have an ancient prog running to deal with our standing order service, that runs under DBaseIII. I know with that, that once records reach a certain level, the software starts throwing a wobbly due to the size of the indexes.

I'm no database expert, but maybe Access has similar problems ie it may allow a database to be up to (I think) 2GB, but may not like it when individual tables or indexes within the database grow beyond a certain size.

Any Access boffins care to comment?

At the end of the day, Actinic isn't really designed to handle mega sized products catalogues, but I've ploughed enough effort into it so far, that it will take a lot to make me change.

[jxm28788]

Just tried (without success) to get a db of 19,750ish products up onto los servers. I can get 50k live no problem as long as I have 40-50 products per page, listed, no image, no description.

I've got 15,000 products, 1 per page, usually with about 15 custom vars each (out of a total of 89 different customvars!), a refresh takes about half a day and the DB is 178mb compacted although I've seen it go well over 1gb uncompacted... Do I win a prize?

[los_design]

I've got 15,000 products, 1 per page, usually with about 15 custom vars each (out of a total of 89 different customvars!), a refresh takes about half a day and the DB is 178mb compacted although I've seen it go well over 1gb uncompacted... Do I win a prize?

I wonder if it is an install issue then?

Back to the drawing board....

[Duncan Rounding]

...Do I win a prize?

You will if you let it go over 2Gb and then find you can't compact it any more!

[los_design]

2.2 Million Product So Far

Hi Darren

megacity currently weighs in at 80MB plus when compacted, and uploads fine, although I am looking at the prospect of splitting into two stores, as I would like to change a lot of the products to one per section, which will eat up any available space I have.

I'm still with Act 5 (despite purchasing 7 & 8 - I just love giving Actinic my money!), so maybe that has something to do with how large db uploads are handled!

If your MySQL test is with Act Enterprise, I'd be interested in your findings, although I do hate those 'enquire for pricing' products, as that often means 'expensive'.

Progress thus far Martin is that I now have 2.2 million products available in Actinic...

I have configured a search that plugs directly into a remote database and now building an XSL fragment to incorporate a remote XML feed. From this, I will implement an XSLT to include via some php the add to cart from anywhere function...

The search is on the live site but the xsl is currently on my testing environment, will shout when I hit that eureka moment.

But fiddly for general use so maybe the SQL route is best for most but this may solve my headache?!

[fleetwood]

I have configured a search that plugs directly into a remote database and now building an XSL fragment to incorporate a remote XML feed. From this, I will implement an XSLT to include via some php the add to cart from anywhere function...

... well thats kinda gone over my head a bit , but glad to hear you are making progress..

<wanders off to wikipedia to find out what all those x-things mean

[leehack]

... well thats kinda gone over my head a bit , but glad to hear you are making progress..

I think that was the idea lol.

[TraceyHand]

I think that was the idea lol.

*sniggers*

stirrer

[los_design]

Sorry guys, not meant to go over your head.

As per the W3C...

This specification defines the syntax and semantics of XSLT, which is a language for transforming XML documents into other XML documents.

XSLT is designed for use as part of XSL, which is a stylesheet language for XML. In addition to XSLT, XSL includes an XML vocabulary for specifying formatting. XSL specifies the styling of an XML document by using XSLT to describe how the document is transformed into another XML document that uses the formatting vocabulary.

XSLT is also designed to be used independently of XSL. However, XSLT is not intended as a completely general-purpose XML transformation language. Rather it is designed primarily for the kinds of transformations that are needed when XSLT is used as part of XSL.

Or, I am developing the idea of building my products into a remote XML feed that will include the add to cart function mentioned and parse the whole gubbings into Act without having the local issues associated with a bloated catalogue. Difference to a MySQL, SQL App is that my aim is to be able to update remotely without the need to upload/update locally.

HTH