Talk:Online DVD Store

CS290F Fall 2006 - UCSB Computer Science - Thorsten von Eicken

Project Comments

Looks good. The database layout is very well thought out, and the diagram is a nice plus.

My only concern is that you don't have any obvious atomic/transactional functionality. That is a necessary requirement for making parts of the fourth project meaningful. So even if it is contrived, you'll have to come up with some atomic action that requires transactionality.

Otherwise it looks great.

Stefan 14:10, 27 October 2006 (PDT)

• The site is excruciatingly slow. Why?
• I tried to register with a short password and got the following error: "1 error prohibited this myuser from being saved There were problems with the following fields: * Password should be at least 5 characters". Note the awkward "myuser". Also, the form was blank: why is the data not preserved?
• I successfully registered leaving the password confirmation field blank.
• The review submission has a subject field that gets ignored.
• You have not yet implemented checkout, it seems. You need to at least implement some gathering of fake credit card information and have a confirmation. Also, where can I see the items I have purchased?
• Queries including string search, such as "(title like "%girl%")" are not indexable. For the purpose of the becnhmarking I recommend you use prefix matching only and build an index.
• You need to use SQL's EXPLAIN statement to see the query plans and figure out which indexes to add.
• Which is the worst query? How long does it take. How can you optimize it?

TvE 17:09, 17 November 2006 (PST)

• Congrats on being early with your first round of performance measurements!
• It looks to me like you have a problem: the lines are too flat... The man page claims that --num-conns=X doesn't apply to the --wsesslog workload generator. I have been using --rate to specify at what rate it should start new connections. You could also use --period (which may be easier to specify. Unfortunately these options make it a bit difficult to predict the number of req/sec actually issued. (You should perhaps vary --period from 10 down to 0.1?)
• You need to show the httperf workload scripts you've been using and calculate (at least roughly) the percentage of "real" Rails requests vs. image requests that Apache serves. The latter really don't count in the "req/sec" figure
• What happened during the optimization of path #3: it became really slow!
• Some of your optimizations change the user interface. You should hold off on doing too drastic changes until the very end. After all, who cares about a fast uninteresting site? :-) You should argue why the UI changes actually improve things. For example, users may now send many more requests your way because they get less information per page. (Ok, they may also send many fewer because they leave your less usable site...)

TvE 07:02, 27 November 2006 (PST)

• Nice measurements presentation and analysis for the single host case!
• Note that your graph legend is mis-leading in that the X axis is really "session started per second" and not "requests per second" (I know you state that in the text).
• What is the rate of real Rails page requests per second (ignoring static & image requests)?
• You state that the DB is the bottleneck: how do you know? Did you use "top" to check?
• For the reviews, you may still want to show the rating on the listing page, just don't show the beginning of the review text (you have a rating, right?).
• Assuming you understand where the bottleneck is, you next step should probably be to break the server into two: one for mysql and the other for apache+rails. Since you have things scripted, this should not be too much work. Now the fun really starts!

TvE 22:55, 28 November 2006 (PST)

Re: Concerning the real rails pages / all requests We dont' actually have any static pages. All DVDs images are fetched from amazon servers (We only store the URL in the DB). That's why we didn't show such measure.

Re: About DB being the bottleneck That information was extracted from Rails log files (/log/development.log) where for each request it shows the time being spent by the DB and the webserver.

Re: Ratings Yes, we could use ratings. Or store reviews captions in separate DB fields to avoid fetching the whole review whenever a caption is to be rendered to the user.

Lamia 19:42, 28 November 2006 (PST)

Re: In addition to using Rails log files, we also analyzed the log files from the database to see which query is taking the longest.

Re: The httperf workload we used was:

/ /browse/list?searchstring=hi&task=search /browse/list?listby=6&task=search&searchstring=hi /?page=2&task=search&searchstring=hi&listby=6 /browse/details?dvd_id=4415&page=2

/ /account/login /account/login method=POST contents='user_name=lamia5&user_password=lamia5' / /browse/list?searchstring=hi&task=search /browse/list?listby=6&task=search&searchstring=hi /?page=2&task=search&searchstring=hi&listby=6 /?page=3&task=search&searchstring=hi&listby=6 /browse/details?dvd_id=8728&page=3 /account/logout /account/login

/ /account/login /account/login method=POST contents='user_name=lamia5&user_password=lamia5' / /shopping_cart/add_item?dvd_id=4382 / /shopping_cart/view_cart /shopping_cart/checkout /shopping_cart/checkout method=POST contents='fullname=lamia5&ccnumber=6767676767676767&address=789tytyGolteta93117' /shopping_cart/confirm /account/logout /account/login