Gui Liberali, Morphing Websites - Digital Data Tips Tuesday #4 - Growth through Data - Amsterdam - May 19th 2015 - #DDTT
•
3 likes•1,109 views
Keynote of the 4th #DDTT meetup (http://digitaldata.tips) in Amsterdam, the Netherlands. Guy Liberali of RSM / Erasmus presented about optimization through algorithms: bandits and morphing websites. Stay up to date about new #DDTT events through our meetup group: http://meetup.com/onlineoptimizers Organizer: Webanalisten.nl / Online Optimizers meetup group Sponsors: Adobe and Relay42
Recommended
Recommended
More Related Content
Similar to Gui Liberali, Morphing Websites - Digital Data Tips Tuesday #4 - Growth through Data - Amsterdam - May 19th 2015 - #DDTT
Similar to Gui Liberali, Morphing Websites - Digital Data Tips Tuesday #4 - Growth through Data - Amsterdam - May 19th 2015 - #DDTT (20)
More from Webanalisten .nl
More from Webanalisten .nl (20)
Recently uploaded
Recently uploaded (20)
Gui Liberali, Morphing Websites - Digital Data Tips Tuesday #4 - Growth through Data - Amsterdam - May 19th 2015 - #DDTT
- 1. Morphing Theory and Applications to Online Experiments: Learning While Earning DDTT 2015 Gui Liberali Erasmus University Laboratory for Optimization of Digital Experiments @ Erasmus
- 2. Morphing
- 3. One size does not fit all (a) General content, large-load, graphical morph (b) Focused, small-load, verbal morph Visual Technical Content More Content Audio Less Content General Content Website designs as Morphs Appeals to Verbal Cognitive Style Appeals to Visual Cognitive Style
- 4. Appeals to Holistic, impulsive cognitive style Appeals to Analytical, deliberative cognitive style One size does not fit all Banners as Morphs
- 5. • From learning-then-earning to the world of learning-while-earning • Higher CTR, conversions • Better user experience Where’s your A/B Testing ? • From aggregate to disaggregate data • From demographics to cognitive styles
- 6. 1. Learning about Styles vs Morphs Firm Side: Banner Ads Buy! versus Learn More Inform a4ve Emo4o nal Time-‐ pressu re Consumer Side: 4 Cogni4ve Styles Verbal-‐Impulsive 0.1 0.11 0.2 0.13 Visual-‐Impulsive 0.11 0.2 0.13 0.1 Verbal-‐Delibera4ve 0.2 0.13 0.1 0.11 Visual-‐Delibera4ve 0.11 0.2 0.13 0.1
- 7. Optimization: from Earn to Learn • Maximize conversion by picking the best among ads for current user • If we knew the conversion probabilities perfectly we would simply serve the optimal morph • If we knew nothing about the conversion probabilities we would randomize and learn Need to balance learning the true cognitive style (exploitation) and providing the optimal morph given what we know now (exploration). For each person!
- 8. Morphing: real-time adaptation of the firm’s actions to each consumer Cognitive-style, segment, position in the purchase funnel, .. Website design, banner, advertising campaign, media copy per channel, promotion, price, product bundle, product recommendation … Consumer latent variable Marketing instrument to be adapted (morphed) x
- 9. Morphing - Evidences from the Field Morphing Banner Advertising (2014) • Morphing almost doubled click-through rates for context- matched banners (83%, p<0.01 and 97%, p=0.028) lifts, respectively, for banners and for consumers • Was live on CNET.com (8.4M visitors/day) for a month • Extends behavioral targeting to avoid local max solutions Website Morphing (2009) • Online sales of BT group’s broadband plans can be increased up to 20% by adapting the website to match cognitive styles using our adaptive learning algorithm Website Morphing 2.0 (2014) • NPV reward improvement of 69% over the NPV of the original algorithm; corresponds to $17.5 million for BT group if implemented • Proof-of-Feasibility on Suruga Bank • Generalized version of morphing, including when to morph, switching costs, multiple morphs
- 10. Morphing Algorithm Challenges Addressed 1. What is the style of the current user? – Update our beliefs as new clicks become available 2. What is the optimal morph for each style? – Update as we observe successes and failures
- 11. Morphing Choice descriptor Priors on distribu5on of styles Updated user style descriptor On-‐line Op4miza4on Pgm: probability of purchase for individual in style g given morph m Banner Ads -‐> 1 2 3 4 Cogni4ve Style Impulsive Delibera4ve 0.1 0.11 0.2 0.13 Illustra5ve example User Op5mal morph for current user Website User makes a purchase or quit Bayesian update
- 12. Optimal solution with Gittins’ indices (assume we know cognitive style) Gittins’ indices for the eight morphs. Morph that was chosen. 0 1 2 3 4 5 6 7 8 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Visitor ChosenMorph 0.35 0.45 0.55 0.65 0.75 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Visitor Gittins'Indices Morph 0 Morph 1 Morph 2 Morph 3 Morph 4 Morph 5 Morph 6 Morph 7 System experiments with Morph 3 for a while before settling back to Morph 2.
- 13. • What is the cognitive style of the current user? – Update our beliefs as new clicks become available • What is the optimal morph for each style? – Update as we observe successes and failures Website Morphing (2009) • How to handle repeat visits and cumulative learning ? – Field experiment on cnet.com w/ AT&T banner ads – Roll-back mechanism Morphing Banner Advertising (2014) • Generalized Morphing Algorithm: w/ When to Morph – DP solves trade-off between gains from morphing early (longer exposure) and gains from morphing later (better knowledge) – Exposure to multiple morphs • Attribution problem: recency, exposure, both • Switching costs: changing morphs only if expected rewards exceed penalty for switching Website Morphing 2.0 (2014) Morphing Trilogy
- 14. Morphing in the Field • Can be used normatively (e.g., field experiments) or descriptively (secondary data) • Two tuning parameters makes it flexible to various applications – Switching costs – Different attribution theories/regimes • Morphing is appropriate to contexts with – High volume of observations – Clear success/failure outcome – Latent variable
- 15. Laboratory on Optimization of Digital Experiments (LODE) @ Erasmus University • Executive education course on Digital Experimentation and Big Data • Research Fridays: Bi-monthly meetings of academics and founding companies at Erasmus University • Bi-yearly publication of research priorities for research on A/B testing • Erasmus students running A/B testing & morphing with real-world company traffic – Module 1: Text-analysis methods – Module 2: A/B experiments – Module 3: Morphing
- 16. References • Hauser JR, Urban GL, Liberali G, Braun M (2009) Website morphing. Marketing Sci. 28(2):202–224. • Urban G., Liberali G., MacDonald E., Bordley R., Hauser J. (2014) Morphing Banner Advertising, Marketing Science, 33(1): 27-46. • Hauser J., Liberali G., Urban G. (Summer 2014) Website Morphing 2.0: Switching Costs, Partial Exposure, Random Exit, and When to Morph. Management Science.
- 17. Thank you! Contact: liberali@mit.edu and liberali@rsm.nl