Speeding Up 'Web Refresh'

[madmatt95]

The website I am responsible for is huge. Last time I did a web refresh it took the best part of 2 days - and that was 6 months ago. (we almost have the best broadband upload speed in the world)

Question (would this work):

1: Before a webrefresh - what if I downloaded all the images inside the acatalog folder / or duplicated folder on server and called it acatalog2

2: Clean acatalog2 folder and leave only .gif, .jpg and .pdf files inside

2: Start web refresh

3: After Purge, upload / copy all remaining items in acatalog2 to the acatalog directory.

When the web refresh deals with the .gif .jpg and .pdf files it should skip them as they already exist, right?

This should at least half the time taken to complete a web refresh.

Does anyone know if this would help me minimise downtime?

[RuralWeb]

If you upgrade to V8 then you have the option to skip unchanged image files and so speed up a refresh.

[Mike Hughes]

Alternatively, do an 'update' as this only uploads the things that have changed.

Mike

[pinbrook]

why do you want to do refresh (replace everything) rather than update (update changes only)?

[NormanRouxel]

I don't think that your steps will work. Actinic creates a list of the files it uses on the live site. Update changes only those that don't match the previous list (or have been deleted from the server).

Refresh is a command to re-upload that entire list - even if they're already there.

Two days seems like a ridiculously long amount of time for any operation.

Is the delay in processing or at the actual file upload stage? Actinic gives a good step-by-step report of what it's doing.

How many products / sections do you have?

What is the size of Acatalog.mdb?

How much storage are you using on the server?

[RuralWeb]

The website I am responsible for is huge. Last time I did a web refresh it took the best part of 2 days

Just had a look and it does not look any bigger than some sites I run and they only take a couple of hours max. Try compacting the database.

[madmatt95]

Is the delay in processing or at the actual file upload stage? Actinic gives a good step-by-step report of what it's doing.

Abit of both really - the purge stage would be at least 8 hours - mainly down to the amount of image we have. Our policy (where possible) one product per page. Every section has a thumbnail, every product has a thumb and main image.

What is the size of Acatalog.mdb?

Fully Purged and Compacted - about 100mb

How much storage are you using on the server?

We are using 992mb of our space (we have a dedicated server)

[madmatt95]

Just had a look and it does not look any bigger than some sites I run and they only take a couple of hours max. Try compacting the database.

Are you sure? Which site do you run that compares to the size of ours?

[madmatt95]

why do you want to do refresh (replace everything) rather than update (update changes only)?

Twice in the past, due to cgi errors - I have had no choice but to do a web refresh (on the say so of actinic support team). Trust me - I realy don't want to be doing a web-refresh but in the future I know I will have to at some stage and am just trying to 'cheat actinic' into a faster web refresh

[NormanRouxel]

Wow! That 992Mb divided by 15000 comes in at a reasonable 66Kb per product.

The time taken up by the Purge seems excessive. It could be that Actinic is making a to/from transaction to the server for each file that it is deleting. A comment from Actinic or someone else may help here.

One way around this might be to manually delete all Actinic content from the server then do a Refresh. Since there would be nothing there to delete, that phase might be a lot quicker, the Refresh actinic much like an inital site upload.

However you probably won't want to be offline for the time it takes to upload everything, making such test rather hard to perform.

As mentioned earlier, V8 allows you to opt out of uploading unchanged images and digital download files when doing Refreshes.

[madmatt95]

If you upgrade to V8 then you have the option to skip unchanged image files and so speed up a refresh.

This is really interesting - we have been reluctant to upgrade because of the problems that I have outlined at the start of this thread. The V8 Option you talk about is great news.

Are you sure this unchanged image files feature is on the web refresh and not the update? Reason I ask is that when you do a web refresh in V7 it PURGES THE WHOLE SITE. Therefore - you are saying this feature does not purge image files, right? Does this also refer to the links tab where you have an internal link say to a PDF?

Any other redeeming features of V8 that would help in ongoing developement speed?

[madmatt95]

One way around this might be to manually delete all Actinic content from the server then do a Refresh. Since there would be nothing there to delete, that phase might be a lot quicker, the Refresh actinic much like an inital site upload.

Thanks Norman - this makes sense. I will contact actinic support to see if they agree with your approach - which i hope they do.

Norman - what about the files that store the 'order number sequence' and all the other cgi information etc - I would not want to loose such files. Would it make sense to refine your suggestion to removing only the following files from the acatalog directory (like in my intial post).....

Catalog files
html files
gif files
jpg files
pdf files

[los_design]

Just thought I would add our perspective on this as we manage a site that holds more than the 15k stated and is in V8 (for you to compare)

With our db currently storing 18765 products (35k next month ) the following timings are what we currently experience.

We have an extended page per item, its own image etc...so not too dissimilar to yours in V7

Site refresh - 13.5hours
Site update - 2-4 hours

DB regularly compacted, files trimmed where poss and we delete all remote content when we need to do a site refresh with no issues.

When using a db of this size, proceed with caution for the also bought list in V8. This can add 200% to the upload time when it needs to generate...

HTH

[RuralWeb]

Are you sure this unchanged image files feature is on the web refresh and not the update

YES otherwise

[RuralWeb]

Are you sure this unchanged image files feature is on the web refresh and not the update

YES otherwise i would not have suggested it