A Framework for Personalization:Susan DumaisMicrosoft Researchsdumais@ Workshop: June 18-20, 2001When do you want to go Where Everybody Knows Your Name (and mailing address, and preferences, and last 50 web pages visited)? A Working DefinitionnOutcome(t) = f(Action(t), PersonalHistory(t-n))nExamples,nRelevance feedbacknContent-based filteringnCollaborative filteringnCaching, history lists, auto completion, MRUnImplicit queries, Rememberance Agent, Watson, KenjinnMyYahoo!, MyAOL, MyMSN, MyLibrary, etc.nAltaVista’s MySearch, iLOR…June 18, 2001Delos-NSF Workshop A Demonstration: What do you see? June 18, 2001Delos-NSF Workshop Many Kinds of Individual DifferencesnTask – “info need”nShort-term, relevance feedbacknLong-term, content-based filteringnPreferences, e.g., CFnExpertise, domain and application nCognitive aptitudesnVerbal, spatial, reasoning skills, etc.nDemographicsnAge, major, gender, location, etc.nCognitive styles, personality and affectJune 18, 2001Delos-NSF Workshop Individual Differences Are …nLargenSystematicnSystems can often be modified to accommodatenE.g., robust systemsnE.g., personalizationJune 18, 2001Delos-NSF Workshop How Big Are Individual Diffs?nE.g., Web searching (Chen & Dumais, CHI 2000)n74 participants; Intermediate web/search experiencen30 search tasks (e.g., Home page for “Seattle Weekly”)nAverage RT (seconds) = 52.3 secondsnIndividual subjects’ average RT: n69, 30, 76, 48, 29, 68, 69, 49, 75, 62, 64, 69, 26, 89, 50, 44, 54, 35, 39, 30, 71, 56, 28, 59, 36, 67, 93, 37, 39, 49, 28, 89, 37, 36, 31, 47, 66, 62, 51, 30, 40, 38, 31, 70, 37, 36, 36, 88, 41, 50, 84, 68, 42, 58, 34, 25, 23, 22, 41, 62, 35, 41, 41, 60, 36, 56, 78, 144, 43, 58, 58, 45, 38, 115June 18, 2001Delos-NSF Workshop Characterizing Indiv DiffsnHistogramnMax:Min144, 22 = 6.5:1nQ3:Q166, 36 = 1.8:1nSD/X.42June 18, 2001Delos-NSF Workshop Example Individual DiffsJune 18, 2001Delos-NSF Workshop Individual Diffs Correlated w/ Performance in HCI/IR TasksnExperience – both application and domainnReasoning (Egan et al.; Card et al.; Greene et al.)nSpatial abilities (Egan & Gomez; Vicente et al.; Stanney & Salvendy; Allen)nAcademic major (Borgman)nVerbal fluency (Dumais & Schmitt)nReading comprehension (Greene et al.)nVocabulary (Vicente et al.)nAge (Egan et al.; Greene et al.; Konvalina et al.)nPersonality and affect nGenderJune 18, 2001Delos-NSF Workshop Framework for Identifying and Accommodating Indiv Diffs nAssay – which user characteristics predict differences in performance; many studies stop herenIsolate – isolate the source of variation to a specific sub-task or design componentnAccommodate – do something about itnOften harder than you think … nE.g., Spatial ability and hierarchy navigation nE.g., ExpertisenEvaluate!!!June 18, 2001Delos-NSF Workshop Greene et al. No IFs, ANDs, or ORs: A Study of Database QueryingnTask: Find all employees who either work in the toy department or are managed by Grant, and also come from the city London.nSQL – fixed syntax, logical operators, parenthesesnE.g., SELECT NameFROMEmployeeWHERE(Department = ToyORManager = Grant)ANDCity = LondonnTEBI – just need attribute names and values; recognize alternatives from system-generated tablenE.g., Name, Department = Toy, Manager = Grant, City = LondonJune 18, 2001Delos-NSF Workshop Greene et al. (Assay)nAssessed individual characteristics: nAge, spatial memory, reasoning, integrative processing, reading comprehension & vocabularynFound large effects of:nIntegrative processing (on accuracy, for SQL interface) nAge (on time, for SQL interface)June 18, 2001Delos-NSF Workshop Greene et al.June 18, 2001Delos-NSF Workshop Greene et al. (Isolate)nExamined two possible sources of difficultiesnInterpreting the querynSpecifying the query in a formal notation or query languageJune 18, 2001Delos-NSF Workshop Example TEBI TableJune 18, 2001Delos-NSF Workshop Greene et al.June 18, 2001Delos-NSF Workshop Greene et al. (Accommodate)nSQL – hard, especially for some usersnTEBI – new query specification languagenImproved performance overallnReduced many dependencies on reasoning skills and agen“Robust interface”June 18, 2001Delos-NSF Workshop How to Accommodate?nRobust interfaces: A new design improves the performance for allnE.g., Greene et al.’s TEBI interfacenE.g., Dumais & Schmitt’s LikeThese interfacenTraining:nPersonalization: Different interfaces/systems for different peoplenGroup level - E.g., Grundy prototypes, I3R sterotypes, Expert/NovicenIndividual levelnTask (Info Need) levelJune 18, 2001Delos-NSF Workshop Personalization FrameworknCharacteristics for personalizationnExpertise, Task, Preferences, Cog Aptitudes, Demographics, Cog Styles, Etc.nAssay: How specified/modeled?nImplicit, Explicit, InteractionnStability over time?nLong-term, short-termnAccommodate: What to do about it?nMany ways of accommodatingnEvaluationnBenefits of correct assessment and accommodationnCosts of mis-assessmentJune 18, 2001Delos-NSF Workshop Content-Based FilteringMatch new content to standing info neednAssay: nExplicit or Implicit profile specification?nOngoing feedback?nHow rapidly does profile it change?nAccommodate: nMatch profile against stream of new docsnReduce number of docs to viewnReturn more relevant docsnBenefits/CostsJune 18, 2001Delos-NSF Workshop June 18, 2001Delos-NSF Workshop ASI ExamplesnCollaborative FilteringnImplicit/Background QuerynLumierenTemporal Query PatternsJune 18, 2001Delos-NSF Workshop nCollaborative filtering algorithmsnBayesian networknCorrelation+nVector similaritynBayesian clusteringnPopularitynTest collectionsnEach MovienNielsennMExample: MSRweb Recommender§ Predicted• Individual scores• Ranked scoreJune 18, 2001Delos-NSF Workshop Example: MSRweb RecommenderJune 18, 2001Delos-NSF Workshop nIdentify content at user’s focus of attention nFormulate query, provide related information as part of normal work flow nBackground, implicit queriesConsider doc structure,basic scroll, dwell patternsExample: Background QueryJune 18, 2001Delos-NSF Workshop Data Mountain with Implicit Query results (highlighted pages to left of selected page)June 18, 2001Delos-NSF Workshop Implicit Query ResultsnFiling strategiesnNumber of categoriesJune 18, 2001Delos-NSF Workshop Implicit Query ResultsJune 18, 2001Delos-NSF Workshop n17 subjects (9 IQ1, 8 IQ1&2)Implicit Query Results(Delayed Retrieval, 6 months) June 18, 2001Delos-NSF Workshop Example: Lumiere nInferring user’s goals under uncertainty*• User query• User activity • User profile• Data structures Pr(Goals, Needs)Pr(Goals, Needs)June 18, 2001Delos-NSF Workshop Example: Lumiere• User’s queryUser’s query • • Sensed actions Sensed actions Atomic EventsAtomic EventsTimeTimeModeled EventsModeled EventsEventSource 1EventSource 2EventSource nEve Event-SpecificationLanguageExample: LumiereJune 18, 2001Delos-NSF Workshop Example: Web Queries 161858lion lions 163041lion facts 163919picher of lions164040lion picher 165002lion pictures165100 pictures of lions165211 pictures of big cats165311lion photos 170013video in lion 172131pictureof a lioness172207picture of a lioness172241lion pictures 172334lion pictures cat 172443lions172450lions150052lion152004lions152036lions lion 152219lion facts153747roaring153848lions roaring160232africa lion160642lions, tigers, leopards and cheetahs161042lions, tigers, leopards and cheetahs cats 161144wild cats of africa 161414africa cat161602africa lions161308africa wild cats161823 mane161840lionuser = A1D6F19DB06BD694date = 970916excite logJune 18, 2001Delos-NSF Workshop June 18, 2001Delos-NSF Workshop Query Dynamics & User GoalsnQueries are not independentnConsider:nSearch goals (e.g., current events, weather)nRefinement actions (e.g., specialize, new)nTemporal dynamicsnBayes net to predict next action, or next search goalnHand-tagged sample of Excite logJune 18, 2001Delos-NSF Workshop Temporal dynamics resultsJune 18, 2001Delos-NSF Workshop June 18, 2001Delos-NSF Workshop Real-World ExamplesnImplicit storage of history of interactionnCachingnHistorynAuto CompletionnDynamic MenusnExplicit storagenFavoritesnMySearch, iLORnRecommendationsnMyBlah …June 18, 2001Delos-NSF Workshop June 18, 2001Delos-NSF Workshop June 18, 2001Delos-NSF Workshop June 18, 2001Delos-NSF Workshop June 18, 2001Delos-NSF Workshop June 18, 2001Delos-NSF Workshop Personalization SuccessnEffectively Assay and Accommodate:nEasy to specify relevant informationnExplicitly: profile changes slowlynImplicitly: capture automatically, esp short timenWe know what to do about itnAlgorithmic and application levelsnAnd, the user can see the benefitnAnd, there are few big failuresJune 18, 2001Delos-NSF Workshop Personalization OpportunitiesnGeo-codingnQuery historynQuery plus usage contextnKeeping found things foundJune 18, 2001Delos-NSF Workshop Open IssuesnEvaluation … difficult for personalized systemsnComponents, easiernEnd-to-end applications, hardernQuestionnairesnPre-Post assessmentnAlgorithmic issues in situnPrivacy, security …June 18, 2001Delos-NSF Workshop The End … June 18, 2001Delos-NSF Workshop 。
