As businesses aim to adapt and sharpen their skills in the digital-first world, they need new abilities to create business success. As customers interact with businesses in a real-time, always evolving competitive environment, they have to implement decisions with adaptive precision and run algorithmic experimentation at granular scale.

Experimentation is the technique carried out under a controlled environment to discover an unexplored effect or principle; to put forward or to establish a hypothesis, or to demonstrate a known principle.

Hence it is very important to create a culture of experimentation within any organization. Experimentation does imply that ideas bloom but also run the risk of some failure, and companies should be prepared for it. Without failing at some ideas, one will never be able to achieve that single great idea that might take your organization to the next level of growth.

Experimentation is an age-old technique to test a hypothesis. A good example of this is retail catalogs, which adopted an experiment to see if prices ending with $0.99, i.e. $7.99 and $8.99, would generate higher sales. To explore the effectiveness of this experimentation, retailers sent out varying product catalogs and, in some cases, left the prices unchanged, and in others, they changed the pricing either at the beginning or at the end of the catalog with $0.99. To their surprise, they found that having such experimentations resulted in an overall increase in sales.

Lee Hibbett, an Associate Professor of Marketing at Freed-Hardeman University, says it might appear ridiculous to price products one cent short of a dollar, but this trick of pricing has a psychological influence on customers. Hence as per Hibbett, because we read from left to right, the first digit of the price resonates well with us most often; hence the reason customers are more likely to buy a product for $7.99 than the same one for $8. This is famously called psychological pricing or charm pricing.

Retailers also use bundle pricing strategies, wherein organizations sell a set of goods with lower prices than they would have charged if the customer bought all of them independently. Most common examples are BOGO (buy one get one free), or buy two products and get another product at 20% off. Pursuing a bundle pricing strategy allows retailers to increase their profit by giving customers a discount.

To get to the optimum products that can be experimented as part of the bundle pricing strategy, retailers rely upon affinity analysis, a.k.a. market basket analysis. The main idea behind affinity analysis is to achieve insights by identifying which products are frequently purchased together. It helps retailers find patterns between purchases in orders, which can be used as a cross-selling opportunity. It can also be used to ascertain what products can go on discount. It’s needless to emphasize that this can apply to many other use cases, which can help increase sales and customer satisfaction.

Another strategy is anchor pricing, which allows retailers to make a product appear cheaper when it is put alongside another product. It is a technique to keep a buoyant pricing strategy. This is the very reason retailers put their own private labels or their own brands next to the market leader, as most often their own brand is cheaper than major label brands.

Price experiments are a technique used by retailers to gauge the association between demand and price change. Price experiments enable retailers to not only present the best price to their customers but also make sure they are rewarded well from a strategic business standpoint.

Tech companies have embraced the experimentation techniques more often than others. Online experiments are vital for ecommerce companies in the development of their web facing products. Given that they have a large user base, even tiny improvements can have a large impact on their profits. For such organizations online controlled experiments are crucial for assessing the impact of product changes in their businesses. They represent the best scientific design for establishing causal association between changes and their influence on observable user behavior.

This example demonstrates how important it is to assess the potential of fresh ideas. A Bing team member of Microsoft in 2012 suggested changing the way Bing search was displaying advertisement headlines, but his product manager considered it a low priority item. It was not until six months later that another team member launched a simple online controlled experiment of an A/B test to assess the impact of it. To everyone’s surprise, within a few hours it was delivering abnormally high revenue.

Similarly, Google’s 41 shades of blue experimentation demonstrates that small design decisions can have considerable impacts on user engagement, resulting in substantial positive engagement.

Digital marketing plays a pivotal role in generating measurable transactions that can be measured to calculate a return on investment. Amazon found that every 100ms of latency cost them 1% in sales. Similarly, a brokerage firm found that if they are 5ms behind their competition then they could lose $4 million in revenues per millisecond.

Thus, businesses are starting to appreciate how critical it is not only to run as many experiments concurrently — but also as cheaply — as possible. Now digital marketing plays a pivotal role in generating measurable transactions that can be measured to calculate a return on investment.

The other decision is the timeframe an experiment has to run for. In general, it is recommended to run an experiment for one to two weeks. Hence the treatment effect measured for such a time frame is called short-term impact. But there are cases when long-term impact is a lot different than short-term impact. For example, increasing the price of products on an ecommerce platform might increase revenue in the short term, but eventually act against long-term revenue because users might shift to some other site for better price.

Experiments might fail due to various reasons, for example the tactics being inadequate. If the experiment fails, then there could be a problem with the underlying experiment design, infrastructure, data or analysis of the result.

Leading global companies competent in algorithmic customer engagement are now powering digital strategies through their products and have always nurtured experimentation as a de facto standard. These companies have embedded experimentation into their organizational DNA, for all their deployment strategies, be they DevOps or MLOps adapts Blue/Green, Canary, A/B testing technique, while rolling out new feature functionalities.

They leverage multivariate testing to evaluate the impact made by their recommendations, merchandising rules and location of one or more placements on certain page(s) on their customers’ ecommerce platform. Likewise, they are also leveraging Contextual Multi-Armed Bandit to intelligently learn from multiple contexts like browsers, user segments, products and handheld devices, for true real-time decision-making supporting personalized recommendations in a seamless and automated fashion.

(This article has only focused on experimentation and how experiments influence decision making processes in the digital-first world and a few techniques that are adhered to in industry. It is in no way a full-fledged representation of various approaches that are followed across the industry since this is an evolving space.)

The article first published in RetailTouchPoints

Based on real eCommerce and marketing examples that tell us what not to do

I thought ‘Hi First_name’ was the worst nightmare for a marketer who’s trying to seem like they have really crafted a message exclusively for the recipient. I was wrong. I hadn’t come across these epic fails that not only cost a conversion or sale, but can negatively impact brand perception.

Superficial personalization is common, but most customers can see through it and find it annoying at best and offensive at worst. Real relevance, on the other hand, is constantly learning and truly individualized. It requires a nuanced understanding of customer preferences, and results in a frictionless, enchanting shopping experience that drives repeat purchases and customer lifetime value.

For a men’s sweatshirt, the recommendations on product page (PDP) include women’s handbag, women’s jacket, and kids pants.

The likely cause for this is using a traditional, common ‘people who bought this also bought’ strategy that clearly isn’t working, as purchases made over time are unlikely to be complementary. Instead ‘bought together in the same order’ is a strategy more suited for compatible cross-sell.

When I look for ‘cutlery’, the webstore returns a no-results page, even though they have cutlery in stock.

The problem is that dictionaries and synonyms aren’t set-up, and the search engine isn’t learning from actual shopper searches. The catalog uses the terms ‘silverware’, ‘forks and knives’ to refer to the same product.

This is a common problem across retail verticals.
In another instance, a search for ‘flip flops’ gets me relevant results, but it does not work when I search for ‘slippers’ or an Australian searches for ‘thongs’ (Yes, I spent some time in Australia, and that’s their term for flip flops!).

I’m inundated with emails, messages, notifications that I never end up opening.

Many marketers are stuck in the 90s – they either lack the ability to target customers, or are constrained because their channels work in silos. Most have progressed to digital channels, but are blasting every communication to every customer on every channel – email, app push notifications, direct mail, SMS. Their inability to cap the total number of touches is of course overwhelming for the customer, and the result is them hitting unsubscribe, turning off notifications, and blocking.

We’re all for relevance and tailoring experiences for an individual’s specific needs. However, know that there is an optimum level beyond which personalization can become restrictive and prevent exploration, giving the impression that there isn’t anything new to consider.

One of our clients found their sweet spot at 65%, and discovered that increasing personalization to 70% on their commerce store resulted in a 5% drop in revenue per visitor and 6% lower average order value.

Read more about over-personalization and findings from testing by a fashion marketplace

Wondering if it’s possible to get customer experience right even as you deal with expanding catalogs, complex shopper journeys, demand and supply shocks, and shorter attention spans? We’re here to help.

Get this made-for-retail personalization idea library to learn about new use cases that are helping retailers and brands improve digital conversions, grow basket sizes, and boost engagement.

If you’re as picky as I am, you probably spend a lot of time researching several products to ensure you make the best decision. But when was the last time an eCommerce store showed pertinent information that helped you make a purchase decision?

This happened to me some time ago when I was on an online store to find a pair of noise-canceling headphones. I used the search bar and was pleasantly surprised to see a headphones comparison article pop up, in addition to relevant headphones. Thanks to the timely and relevant article that showed up, I was able to speed up this process.

Most businesses think of eCommerce personalization only in terms of product recommendations. However, it’s equally important to personalize other key touchpoints in the online shopping journey — search, browse, and content. Doing so allows for a more holistic customer experience, which translates to increased conversions and revenue per visit. eCommerce search is increasingly becoming a key pillar of CX.

Fig 1. Touchpoints across the eCommerce shopping lifecycle

Why? For one, shoppers who search convert 3x more than those who don’t. This has to do with the fact that people who use search have an express intent of what they want to buy.

Personalization search ecommerce should kick in as a shopper types their search query — showing products that they are more likely to be interested in based on their individual behavior and known preferences. But before getting to personalized results, the basics need to be in place, and this requires effort and is not without its challenges.

Let’s look at some common challenges with traditional search engines through the eyes of one of our clients: Verkkokauppa.com. Verkkokauppa.com is Finland’s most-visited online retail store with 65,000 SKUs in 26 product categories.

When a shopper searches for ‘Apple’, the results could display every Apple product available. But is that relevant to the shopper? Perhaps not. One way of addressing this is to look at the collective behavior of shoppers after they searched for ‘Apple’ and tune the results accordingly. This is the essence of what’s known as Wisdom of the Crowd (WoC).

WoC uses a Machine Learning algorithm to learn from the entire shopper population, their queries, and what they view or purchase subsequently. And the results are, more often than not, accurate.

In a recent conversation, Verkkokauppa.com shared how leveraging WoC helped them bring relevance to the results when the search queries were as short as one word.

Take the example of the word ‘lamppu’, which is Finnish for a light bulb. The issue with the word ‘lamppu’ is that it’s also used as a compound word for several categories of lamps — such as ‘taskulamppu’ which means a torch, and ‘älylamppu’ which refers to a smart lamp.

Figure 4 below shows the search results for ‘lamppu’ before the application of WoC. The results show various types or categories of lamps because the product titles contain the word ‘lamppu’. While the results are technically correct, they aren’t relevant.

Fig 4. Search results for ‘lamppu’ before implementing WoC

Figure 5 shows WoC in action. The search engine now knows that people using the search query ‘lamppu’ are in fact looking for a light bulb. So it displays several options for a light bulb at the top of the results and pushes the other categories of lamps below.

Fig 5. Search results for ‘lamppu’ after implementing WoC

The WoC functionality, therefore, helps understand shopper terminology like a human would, and boosts relevant items, while eliminating the dependency on developers to build various rules.

Leveraging behavioral data to drive 1-to-1 personalization is now table stakes. Search personalization helps convert shoppers with clear purchase intent quicker.

The retailer uses ADA Global Find™, an ecommerce personalization software that not only helps deliver richer personalization but also improves product discovery optimization, with the use of behavioral data. It leverages behavior attributes (see Fig. 6) to organize search results, as per the individual’s intent, in real time.

The personalization software allows for granular controls that help set weights and experiment with each of these attributes. This is as simple as moving sliders. The software, therefore, empowers business users to adjust personalization settings themselves as and when needed without any dependency on IT.

Fig 6. Personalization tool settings to adjust behavioral attributes

In a world where consumers interact with digital commerce even when the actual purchase is made in-store, it’s more important than ever for retailers and brands to invest in customer experience strategically, and across the entire shopper journey.

While devising an ecommerce search personalization strategy, retailers must think beyond product recommendations and recognize that search is no more just a developer tool, but a lever for business growth.