Storage device performance prediction with CART models(12)

Storage device performance prediction is a key element of self-managed storage systems and application planning tasks, such as data assignment. This work explores the application of a machine learning tool, CART models, to storage device modeling. Our appr

Predicted response time (ms)

1600 1200 800 400 0

100 %80 %% of requests

60 %40 %20 %0 %

ActualPredicted

1000 2000 3000 4000

Response time (ms)

400 800 1200Actual response time (ms)

1600

(a)Scatterplot(b)Responsetimedistribution

Figure5:Predictionaccuracyoftherequest-levelmodel.Theactualandpredictedaverageresponsetimesare137.96msand133.01msrespectively.Thecorrespondingdemerit,de nedin[28]astherootmeansquareofhorizontaldistancebetweentheactualandpredictedcurvesin(b),is46.06milliseconds(33.4%oftheactualaverageresponsetime).

effectiveincapturingrequest-levelcharacteristicsneededtopredictresponsetimes.

Insummary,therequestdescriptioneffectivelycapturesimportantper-requestcharacteristics,leadingtoaccuraterequest-leveldevicemodels.

5.2ModelingASingleDisk

Figure6comparestheaccuracyofallthepredictorsinmodelinganAtlas10K9GBdiskonreal-worldtraces.Asmentionedearlier,allthepredictorsaretrainedusingthe rsttwoweeksofcello99a.Overall,thetwoCART-baseddevicemodelsprovidegoodpredictionaccuracyinpredictingboththeaverageand90thpercentileresponsetimes,comparedtootherpredictors.Severalmoredetailedobservationscanbemade.

First,allofthemodelsperformthebestwhenthetrainingandtestingtracesarefromthesameworkload,e.g.cello99a,becausethemodelshaveseenhowthedevicebehavesundersuchworkloads.Thepredictoralsocutsthemedianpredictionerrorofthepredictorbymorethanahalfbecauseofthestrongperiodicityoftheworkload.andfurtherreducetheerrorto4.84milliseconds(19%)and14.83milliseconds(47%)respectivelyfortheaverageresponsetimeprediction,and20.46milliseconds(15%)and49.50milliseconds(45%)respectivelyforthe90thpercentile.Theperfor-androughlymeasuresthebene tofusinganon-linearmancedifferencebetween

model,suchasCART,becausebothacceptthesameinput.Weobserveasigni cantimprovementfromtheformertothelatter,suggestingnon-lineardevicebehavior.

Second,bothCART-baseddevicemodelsinterpolatebetteracrossworkloadsthantheothermodels.

andrelyblindlyonsimilaritiesbetweenthetrainingandtestingworkloadstomake

goodpredictions.Consequently,itisnotsurprisingtoseehugepredictionerrorswhenthetrainingandtestingworkloadsdiffer.TheCART-basedpredictors,ontheotherhand,areabletodistinguishbetweenworkloadsofdifferentcharacteristicsandaremorerobusttothedifferencebetweenthetrainingandtestingworkloads.

Third,modelaccuracyishighlydependentonthetrainingworkloadqualityfortheCART-basedmodels.Thepredictionerrorincreasesforworkloadsotherthancello99a,becauseoftheaccesspatterndifferencesamongthesetraces.TheCART-basedmodelslearndevicebehaviorthroughtraining;therefore,theycannotpredictperformanceforworkloadsthathavetotallydifferentcharacteristicsfromthetrainingworkloads.