Originally posted by Chern Ann
I only regret not doing this a year ago
. The lesson here in case anyone reading this has to deal with MySQL is, don\'t update the same table you\'re reading from on every page. The query cache is cleared every time the table is updated, there\'s a huge IO write overhead when writing to a large table that is better handled as a batch rather than per user.
Now data that is being updated is stored in a temporary table before being transferred to the main table in batches. The browse page reads a static cache that is updated every hour instead of the live image database. It looks and feels the same to an end user but is more than 10 times faster.