Storage device performance prediction with CART models(14)

Storage device performance prediction is a key element of self-managed storage systems and application planning tasks, such as data assignment. This work explores the application of a machine learning tool, CART models, to storage device modeling. Our appr

averageresponsetime

(b)predictionerrorfor90thpercentileresponsetime

Figure7:ComparisonofpredictorsforaRAID5diskarrayof8Atlas10Kdisks.

Amodel’serrorconsistsoftwoparts.The rstpartcomesfromintrinsicrandomnessoftheinputdata,suchasmeasurementerror,andthiserrorcannotbecapturedbyanymodel.Therestoftheerrorcomesfromthemodelingapproachitself.TheCART-basedmodelsincurerroratthreeplaces.First,thetransformationfromworkloadstovectorsintroducesinformationloss.Second,theCART-basedmodelsusepiece-wiseconstantfunctions,whichcouldbedifferentfromthetruefunctions.Third,alow-qualitytrainingtraceyieldsinaccuratemodelsbecauseCARTreliesontheinformationfromthetrainingdatatomakepredictions.Aninadequatetrainingsethasonlyalimitedrangeofworkloadsandleadstolargepredictionerrorsforworkloadsoutsideofthisrange.We ndthatthelasterrorsource,inadequatetrainingdata,causesthemosttroubleinourexperiments.

Weconductasmallexperimenttoverifyourhypothesis.Figure8(a)comparesthedifferenceinse-quentialitybetweencello99aandcello99c.Thespectrumofsequentiality(from0%to100%ofrequestsintheworkloadbeingsequential)isdividedinto20buckets,andthegraphsshowsthenumberofone-minuteworkloadfragmentsineachbucketforbothtraces.Weobserveasigni cantnumberofhighsequentialityfragmentsincello99b,butnofragmentgoesbeyond50%sequentialityincello99a.Thisdifferenceleadstolargepredictionerrorsforhighsequentialityfragmentswhenwebuildtheworkload-levelmodeloncello99aanduseittopredicttheperformanceofcello99b,asshownin(b).Theerrorsarereducedsigni cantlywhenweincludethe rsthalfofcello99bintraining.Thedramaticerrorreductionsuggeststhatpredictioner-rorsfromtheothersourcesarenegligiblewhencomparedwiththeonesintroducedbyinadequatetraining.Figure8(c)furthershowstheabsoluteerrorhistogramwith1millisecondbuckets.Thespikeshiftto0millisecondswhenwetrainthemodelonthecombinedtrainingtrace,indicatingthatitisreasonabletoas-sumeazero-meannoiseterm.Weconcludefromthisevidencethatcontributingeffortsinblack-boxdevicemodelingshouldbedirectedtowardgeneratingagoodtrainingsetthatcoversabroadrangeofworkloadtypes.