MeasureCamp Italy 2 took place on 18th of May 2019 in Rome. Resolution attended the conference. We wrote about general MeasureCamp remarks, Intelligent Tracking Prevention session and inspiration here. Here are more insights from the conference.
Frustration rate and confusion rate with Mikko Pippo
Another session I attended was held by Mikko Pippo, a Digital Analytics and CRO Consultant from Hopkins in Finland. He gave a talk on the need of turning measuring success into measuring failure. In his opinion, lost sales or conversions highlights missed potential better, and thus enact change. The catalyst for Mikko’s idea was lack of motivation to improve the website by his client when he reported solely on success metrics.
Mikko customised a Google Analytics setup for analysis of online user pain points. He utilised calculated metrics to express user’s annoyance and confusion probably related to low usability. Hence, the frustration rate was created. The rate is number of clicks divided by number of page views expressed in a percentage. Frustration rate measures user annoyance by registering number of subsequent, chaotic clicks on a single page. The assumption was that an angry user would try to make a website response with brute force. The frustration rate calculated metric can be utilised neatly in a custom report in Google Analytics. When supplemented with a list of page URLs, it marks pages that include malfunctioning elements causing users to click around.
Another calculated metric proposed by Mikko is the confusion metric expressing number of page views divided by the number of unique page views. Unique page views are ordinary views de-duplicated for a single session. The metric demystifies which pages are optimised and which fail to deliver desired information. When a certain URL delivers many page views but 1 unique page view, it may mean that the user is navigating forth and back through a page failing to find the right information. It can also suggest that some pages fail to load correctly forcing the user to click refresh. The confusion metric can be used to optimise different types of pages; from content, through form to e-commerce checkout pages.
The key take-away from Mikko’s session was that the calculated metrics may not be understandable for all stakeholders due to differences in data-literacy. Also, some users may find it difficult to navigate through custom reports. Therefore, he suggested that his custom metrics should find reside in an optimisation dashboard, where the data is ready to access instantly and is updated automatically.
When divided by number of sessions they constitute the irritation rate.
Source: Mikko Pippo, SlideShare
On the other hand, unavailable products, rejected payments and rejected sales are considered macro-anti-conversions due to critical impact on final financial outcome. When divided by the number of sessions these constitute a suck rate.
Source: Mikko Pippo, SlideShare
Mikko also suggested filling out goal values with the financial figures, especially for e-commerce clients. Assigning missed product and transaction values to goals allows to approximate for lost revenue. Since goals are de-duplicated by a session, a user trying to push several transactions within the same sessions is only counted once.
Prospect Theory and goal setting
Mikkos approach to setting anti-conversions is based on Daniel Kahnenman’s Nobel Memorial Prize-winning research on the Prospect Theory. The theory revealed several cognitive biases which affect the decision-making process. Firstly, when provided with a risky choice the study subjects were more prone to choose based on the perceived value of loses and gains, rather than the outcome of accumulated wealth. Secondly, the value of perceived losses triggered risk-seeking behaviour within the subjects, while the value of perceived gains triggered risk-aversion. Finally, the subjects perceived value of losses and gains differently based on the dynamics of the situation. Hence, if the value was decreasing, the subjects perceived it as a more pressing trend, than the value increasing at the same rate.
Concerning Kahneman’s research, changing the elements or structure of a website that mostly runs correctly may be perceived as a risky activity by those who are responsible for online sales. However, an approach to report on lost opportunities and revenue should, in theory, influence the ‘subjects’ to undertake certain measures to fix malfunctioning website elements. Since the focus of implementation is on approximated value of losses it should trigger ‘risk-seeking’ behaviour which, in this case, is a decision to act. In Mikko’s example the issue were finally prioritised and measures to improve were taken.
Prospect theory and Conversion Rate Optimisation
One of the take away’s from Mikko’s approach to the Prospect Theory is how to use it for CRO (Conversion Rate Optimisation) projects. Once anti-conversions and missed revenue are reported on and motivate stakeholders to act, induced sense of risk can be mitigated by a set of CRO activities. These help to identify the best alternative for the website improvement strategy. Running A/B split testing on a checkout funnel helps selecting the right solution to maximise revenue from users who already made a purchase decision.
Source: Amine Khaddar, MeasureCamp, Facebook
Moreover, reporting on missed opportunities to build an optimisation strategy showcases how powerful a mix Web Analytics and Conversion Rate Optimisation services is.
E-commerce funnel visualisation
Mikko’s approach to report on errors, frustrations and missed opportunities is still considered experimental. Also, collections on such a vast amount of data may well exceed the basic Google Analytics account hits limits. A potential solution is to implement an anti-conversion view within a separate property where the failure events do not contribute to the overall hit quota. Separation of data into multiple properties however decreases its usability, as client IDs get fragmented.
A different solution based on the Prospect Theory is reporting on drop-off rates in e-commerce funnel tracking. This solution has a single drawback; most sources wrongly recommend tracking steps with page views instead of events limiting latter data activation and export opportunities. Tracking steps with events does not spoil the reports within the interface, on the contrary, it provides more, richer data for analysis and activation.
There are several steps in structuring event-based approach for commercial data:
- Triggering of all e-commerce actions and views as events
- Setting view-based events to non-interaction to true to preserve bounce rate
- The creation of an e-commerce optimisation view
- Implementation of e-commerce event-based goals; one per step
Apart from using these goals for optimisation campaigns, remarketing and or even triggering emails, one may also export enriched e-commerce data to visualisation tools. Additionally, having the steps run as singled-out metrics enables calculation of drop-off rates in between all e-commerce steps, including checkout.
With the data configured in a visualisation tool, an e-commerce optimisation dashboard can be made. The whole funnel can be sorted by a wide range of session and e-commerce dimensions. Product type or product brand together with channel can bring valuable insights on user purchase preferences based on the acquisition channel. Using customised dimensions, such as full referrer can also shed new light on performance or lack thereof from gateways providers.