Moving the North Star Metrics for GeekTrust;

19 min readMar 28, 2021

Problem Statement;

Currently, our monthly code submissions average around 100 to 150.
We want to improve this, to an average of 1000 code submissions per month. To achieve this goal, we want to understand how you would approach:

The Go to Market strategy.
Product changes needed to meet this goal.

I am dividing this assignment into two parts. They are:

Problem Space: where I would like to break the Problem in-depth in order to understand the right Problem which needs to be Solved.
Solution Space: Try and address the identified problem to achieve the desired Goal.

Understanding Company and Market;

GeekTrust is a Hiring Platform for Developers. Here is how it works:

Understanding Problem Space better;

Well, this is just a list of questions, I had asked before starting with the Problem Statement. This helped in better understanding of the Problem statement and narrowing it down to the Core Problem:

Who are our potential users and how would their persona look like? Can I even look beyond them? — From our data, the majority of our users are in the 1 to 8 years experience range and are from a tier 2 background (college/employer), and are looking to switch to an aspirational company.
"We want to improve from 100-150 submissions/month to an average of 1000 code submissions per month" - Do I have any specific timeframe as in by when we are projecting to get an average of 1000 submissions/month or Should I assume? — We would want to hit 1000 submissions by the end of the year 2021.
I am assuming 100-150 submissions/month, including all the problem statements that have been put up on the Website. Please correct me if I am wrong (if it's 100-150 submissions/month/problem statement) — This is correct. Across all problems.

Root Cause Analysis - I would like to understand the funnel and figure out the right problem in order to improvise it. As I am short of time, please bear with me and help me understand with numbers:

The number of registered users with us? - Activated — 60,000 out of which 30% are empty profiles
How will a registered user and a non-registered user get to know about the problem statement/hiring when published on the website - channels (if anything other than Email) and what is the share of acquiring them from each channel - Acquisition. — On the website, Emails, phone calls - 90% is emails
What is the percentage of new user vs repeat users who view; download the problem statement - Engage — Will have to check this, you can make an assumption here for now.
What is the percentage of new users vs repeat users who view and don't submit the solution? - Churn — Not sure again, you can assume 50% churn from download to submission. However, we know that only 10% of our total user base submits code.
If a number of submissions are the output metrics/Focus Metrics. What are the input metrics for the same? — Registrations -> Downloads coding problem OR accepts a company pitch -> Submits code
Should I be considering any Budget allocation cap for driving this growth until Dec 2021? — Assume we have a free budget

Breaking down things;

<Metrics that Matters>

I have listed High-level Metrics that Matter the most for the purpose of this assignment. Here are they:

Focus Metrics: Number of Submissions. Input Metrics: Number of Registrations → Number of Downloads of Problem Statements or Accepts Company Pitch.

Decoding the DATA;

Number of REGISTERED users: 60k which 30% are Empty = 40K Actual Registered Users
Number of Activated users: 10% of Actual Registered = 4k accounts [Submitted Solution at least once]
Number of Problem Statement Downloads/month = 250 [ 0.5X* Number of Submissions]
Number of Solution Submissions/month = 125 average.
But Considering 100 repeat users and 25 new users in 125 submissions/month (Assumption).

Report - Per 125 submissions / month from 60k users;

10% conversion rate or 90% churn rate on Number of Actual Registered Users: Number of Activated Users.
50% conversion rate or 50% churn rate on Number of downloads : Number of Submissions.
5% conversion rate or 95% churn rate on Number of Activated Users : Number of Downloads.
2.5% Conversion rate or 97.5% churn rate on Number of Activated Users : Number of Submissions.
20% new user submission each time (Assumed)

The below image represents the current situation from the Report. It's a leaky bucket that can't hold our users Strong in a loop. The assignment here is to fix the bucket and hold things Tight.

Projected Submissions/month = 1000 submissions/month by the end of December 2021.

The solution space is basically divided into two parts. They are:

Recommended Product Changes.
Go-To-Market Strategy.

Before jumping in for solution ie., What product changes to be driven for overall Growth to reach 1000 submissions/month. I would love to understand what growth looks like at different stages in a start-up’s life cycle. So let me walk you through Morgan’s Brown take on the key stages, and related inflection points, in a startup’s growth trajectory.

According to Morgan’s Brown, the key stages of start up’s life cycle are:

Problem - Solution Fit.
Minimal Viable Product.
Product - Market Fit.
Scale.
Maturity.

What Experts have to Say;

Growth is not a Goal, but it's a Process and Don't forget Retention on your way!

Here is what Brain Balfour has to say about both Growth and Retention. He first explains Bad Growth vs Good Growth and below are the graphs that represent the same.

Bad Growth: This type of Growth is something, that looks very good since the days of implementation of various strategies and declines continuously over the time-frame to hit Zero. The amount of Efforts, Resources spent on this type of Growth don't sustain long enough and eventually, the Product fails.

Good Growth: This type of Growth is something, that looks very small at the initial stages but explodes exponentially over a period of time. The amount of Efforts, Resources spent on this type of Growth sustains very long enough and eventually creates Successful Products. These are the type of Products that wins over every other product/competitor in the Market and ends up acquiring a large chunk.

And about the Retention, He says:

"Retention rate is your best measure of product/market fit, the higher your retention rate the more it’s a must-have product"

in other words,

"If Retention is towards ZERO, you Might have a Product Market Fit Problem and Not the Optimisation Problem"

Evidence;

Considering our REPORT that I have created above (includes Growth & Retention Numbers, analysis of Bad Growth, Good Growth). I feel we @GeekTrust are struck somewhere in-between achieving Product-Market Fit stage in start up’s life cycle and needs a Tinkering around it before we move towards optimization.

Ps: As I would not have the complete data, It's really tough for me to either apply both Segmentation or Cohort Analysis and User / Customer Research in order to gain in-depth insights. These insights are based on Hypothetical assumptions.

The above image depicts our problem. We cannot just build great things on Weak Foundation (average Value Prepositions) as it's non-sustainable (for the long term). In order to grow submission from 125/month to 1000/month, we need to make sure the Foundation (Value Preposition to PMF) stands solid at first. Hence,

I am dividing my solution into two parts:

Part 1 - Check and Attain Long Term Product Market Fit - Foundation
Part 2 - Funnel Optimisation - Building

Part 1 - Product Market Fit;

The Product Market Fit (Foundation) test consists of 2 subparts. They are:

A) Value Preposition fit.

B) PMF Optimisation.

A) Value Preposition Fit;

Here I am trying to address Value Proposition Fit for GeekTrust which includes understanding and Optimising Value Prepositions, Features, and Benefits. I am using Dan Olsen's Framework for the same and the below image gives us an overview of the Process.

I created a proto-persona that includes demographics, goals, and behaviors. This helped me start to envision the kind of problems that such a user would have.

This proto-persona was based on a lot of assumptions that required further research for validation, but it was a quick way to guide myself through the challenge.

What did Programmers / Software Engineers say?

First, I listed the desired top outcomes that our proto-persona would like to have, and to understand this better, I gave a quick call to two of my friends in order to understand their perspectives, and here are their Highlights:

As a Programmer / Software Engineer, I want to practice regularly and make a switch to my dream company"
As a Programmer / Software Engineer, I want to learn more and more from time to time.
As a Software Engineer, I want to practice Tougher and Better Problems.
As a Programmer / Software Engineer, I want to actively discuss with people to clear my doubts and get updated about recent industry Trends"
As a Programmer / Software Engineer, I want to get referred to Companies.
As a Programmer / Software Engineer, I want to give Mock Interviews and attain confidentiality.

The kano framework is another excellent framework for understanding customer needs and satisfaction. The utility of the model is that it breaks customer needs into three relevant categories that we can use for our case study. They are Must Have's, Performance Benefits, and the Delighters. Here are what I found:

Must Have's:

Practise Problems
Community
Hiring Challenges

Performance Benefits:

Courses
Mock Interviews

<Market Research>

There are various Direct and Indirect Competitors for GeekTrust. I have divided them into Hiring Platforms and Learning Platforms.

The Direct Competitor for GeekTrust would be Interview Bit as it gives Head - Head competition with the same USP.
The Indirect Competitors, those who give users the advantage of both the Learning Platform and Hiring Platform (Screening Round), are HackerEarth and HackerRank.
The Indirect Competitors, those who give users the advantage for Learning and helps individuals to get Hired are Codechef, Topcoder, CodeForces, ATcoder, Newton School, Upgrad, Udacity, etc.,

Even though the learnings can be taken and adapted from all the platforms (both Learning & Hiring — Direct & Indirect Competitors), I have tried to give a quick overview only about our Direct Competitor (due to shortage of Time):

Mapping it with the GeekTrust set of Value Prepositions, it helps us to understand what drives the better Product Adoption from the Competitor, in the Hiring Market:

While the above Table gives detailed comparisons of value prepositions between Direct competitors ie., InterviewBit with GeekTrust. I pulled my friends back on call again to vote on all the features that made the better product. And InterviewBit clearly wins the competition at Each Stage ie., Must Have's, Performance Benefits, and Delighters.

So my recommendations for Product Changes (Value Prepositions) to gain higher PMF are:

Learn - Fast-track Courses (Hooks users after Registration).
Practise Problems (Hooks users after Registration) - Scores which don't consider for Hiring.
Mock Interview Prep.

Ps: Every recommendation can't be the final solution. Hence I will not be able to discuss the entire list of optimization solutions in detail. Let's do it at the later stage of the assignment.

B) Optimization;

Optimization at this stage means optimizing things that actually contribute to higher Product - Market Fit and not to be confused with Growth Loops. I am Concentrating on the basic things such as Testing Landing Pages, Call to Actions buttons, User onboarding Experience, etc., and the core intention is for optimizing Activation, Conversions, and Retention loops.

Aha Moments / Magic Moments - Actions, Experiences and push users to take those actions or hit milestones - this will drive activation, conversion, and retention.

So my recommendations for Optimisation at this stage (PMF startup cycle) are:

Design changes for Landing Page - Tag Line, Value Preposition positioning & CTA's. (AIDA FRAMEWORK)
Socialize - Follow Fellow Developers, recent participation, solutions, etc.,

Ps: Every recommendation can't be the final solution. Hence I will not be able to discuss the entire list of optimization solutions in detail. Let's do it at the later stage of the assignment.

Part 2 - Funnel Optimisation;

These are the general funnel optimization that we need to drive in post Product - Market Fit, in order to achieve our desired growth and Goal (Building great things on a strong Foundation). They are:

Content Creation and Indexing.
Hire Micro-Influencers to attract more developers/peers.
Consider scores from other platforms - Don't start from First, We accept your cross-platform scores/investments.
Get Great Companies to Hire our Developers (Chicken and Egg Strategy)
Enhance Gamification techniques at various stages - Points, Badges, and Leaderboards.
Tie up's with Learning Platforms and Help them to Hire great candidates - Upgrad.
Increase Number of Hackathons listings - their toughness across Different Categories (Illustrations are Great)
Newsletter and Monthly / Weekly Statistics covering Journeys, aspiring Geek - Developer Stories.
Grow both Internal & External Community on various platforms ie., Youtube, LinkedIn.
Promote our USP CodeU - Clean code importance / Looping it with practice questions and Hiring Questions.

Ps: Every recommendation can't be the final solution. Hence I will not be able to discuss the entire list of optimization solutions in detail. Let's do it at the later stage of the assignment.

Growth Experiments:

We just don't hit the road of Feature Development. There are a lot more things to it before that. Below is the table that helps to understand how we go around and test waters. This indeed helps us to clarify that all our Assumptions / Hypothesis about Feature Prioritisation wrt the desired persona is perfect and this helps to save a lot on Cost, Time, Efforts, Talent and everything that goes in.

The steps for the Growth Process are:

Step 1: Brainstorm in other words means, Generate a lot of ideas on what do we need to work on next. As in what Metric do we have to move in particular. This can come from anywhere, maybe from the organization or from the customers or from someone who never used the product.

Usually everyone before starting any Growth process, this task is shared with everyone to ideally come up with the ideas. Now, Let's start by listing all the recommended features at each stage.

Step 2 - Prioritise in other words mean, Organise and Prioritise. Rate and Compare their potential impact, confidence and resources required. It's very important to acknowledge the fact that not every feature can be implemented considering Time, Effort, Cost, and Resources for any startup. So I am Prioritising Features accordingly that get aligned with our Goal. ie., Increasing Submission/month to 1000 by December 2021.

First things first, In order to optimize things to reach the goal. We definitely have to build Value Prepositions right and achieve PMF. Hence I am considering Learn, Practise Problems, and Mock Interview as constant irrespective of what. The below table is an effort which is made to show how to get closer to our goal.

Post driving Value Prepositions right, We need to Prioritise all the other features that help in optimizing the funnel so that it leads closer to our Goal. Here is the table that actually compares Feature Dev in terms of Value-Driven vs Efforts vs Resources vs Cost.

Ps: Resources and Cost factors have been neglected as I am given a Free Budget and Resources.

The final set of Features/optimizations that are Considered and Roadmapped are shown below in the Table. This has been prioritized keeping few things in my Mind. They are Value vs Efforts vs Cost vs Resources.

Optimal consideration: [ (High Value) vs (Low / Medium Effort) * (Low / Medium Resources) ]

Step 3: Test means Executing all the prioritized ideas in smaller steps. We do Smoke tests and collect Qualitative & Quantitative data around it, usually called Minimal Viable Feature. It's mostly open for targeted personas and a small chunk of the audience ie., Sample Size, and named Minimal Viable Test. It saves a lot of Effort, Time, Cost, and Talent.

Step 4: Analyse meaning Compare results to your hypothesis and ask why. The below table shows an example of how we collect data and Analyse their results at regular intervals of time. This helps to conclude things down, whether to Preserve or Pivot.

According to the table, The waves are the intervals between the time, that we actually test our products with the user in order to understand the Product Feature Adoption and scope for improvement. We indeed measure the Overall Satisfaction score in the regular time intervals to understand how well the features are performing.

Step 5: Optimise meaning use all the leaning that we obtained to improve or execute and systemize the idea.

Step 6: Report which means once the test is completed, go back to plan and write the summary of everything that happened in a loop. If the Test fails, understand why, and if the test wins Go ahead and execute it in full fledge to take it to a larger part of the Audience.

This drafted Report is always can used again as a Playbook before starting a different test again for learnings.

Roadmap;

Post experimenting Features and optimizations that we want to drive, we will go ahead and create a Backlog with the results obtained in the process. So here is a sneak pick as in how our Roadmap looks like for GeekTrust until December 2021:

A Go-To-Market Strategy is a comprehensive action plan to successfully launch a product in the market. Building an effective go-to-Market strategy is an important aspect of the entire product development because It largely determined the success of a product or service when it is launched in the market.

Basic advantages of having a well defined go-to-market strategy are as follows:

Helps companies to understand their target market and target audience better
Enable effective utilization of Time and Resources
Helps companies to minimize their overall cost

A Go-to -Market plan is needed at many stages. Few of them are:

A company is looking to launch a new product in the market.
A company is looking to launch an existing product in the new Market.
A company is looking to launch a new feature of an existing product in the Market.

A. Understanding of the product and the market:

I have already defined what would Ideal Customers for GeekTrust look like — their persona, their pain points, and the competitors around by pinpointing their Strengths and Weaknesses. It’s time for us to think about a strategy to ensure that your product achieves a long-term product/market fit.

Achieving Long Term PMF is a constant and iterative process. Constantly revisiting our understanding of the market, our product offering, and its value proposition, and the messaging is critical to ensure constant PMF where the market and customer’s needs are also constantly changing.

Initial user feedback can be gathered using some of the techniques mentioned below:

Beta launch: Feedback from early adopters of the product / Features.
User voice: Forums where existing customers can provide feedback for the product teams.
Customer interviews: Product teams directly engaging with ideal target customer personas to get feedback.
Market research: Through independent forms, analysts and industry surveys.
A/B testing: On the website, to see how different visitors interact with the product/platform.
Different sales and marketing channels: Feedback from the different marketing and sales channels used to run different experiments.

Beyond the initial product-market fit, the company can continue to use the following to gather ongoing customer feedback. This feedback should be effectively routed to the appropriate teams (Product, Marketing, Sales, Support) such that changes are made to ensure fit for the long term. There is also a need for ongoing research to understand market trends so that the product can be tweaked accordingly. In some cases, when a company introduces new products, it could potentially change/ expand the customer profile.

B. Sales and marketing strategy

Marketing Channels:

Paid advertising: Using paid ads on popular social media channels and portals that are frequented by the relevant audience from the developer segment would be a good idea. These could be channels such as LinkedIn, Facebook, youtube, and Twitter.
Email marketing: This requires that we build a database of email addresses over time that can be targeted through nurture campaigns. The initial list can either be purchased through database vendors specifically targeting developers or making the email ID mandatory as part of the initial sign-up process. Tools such as the LinkedIn Sales Navigator also provide the ability to directly email the right folks in target developers based on the industry, geography, and company size.
Social media: A company’s social media channels (including LinkedIn, Facebook, Twitter, and Instagram) are great marketing channels to tell your story, offer engaging content (in the form of blogs and videos), explaining the product, its use-cases, customer testimonials. They also allow you to occasionally engage users through games/quizzes and other means of social engagement.
Referral program: A referral program that encourages word-of-mouth marketing and incentivizes the referrer could be another channel to drive initial adoption. The referral program should ideally help acquire more customers and build brand awareness at lower CAC (customer acquisition costs); so, the incentives should be structured accordingly. This strategy also increases the product reach to users who may not have been targeted through some of the other channels.

Some companies also effectively use content marketing by providing useful content regarding team communication and collaboration, by particularly targeting the developer audience.

Sales Channels:

Social media: In addition to being a great marketing strategy, a company’s social media channels can also be used to sell. When interested users land on the company’s social channels, it is important to effectively engage them with the right content to help them move to the Website and signup with us. Many customers also prefer direct messaging on these channels. Having a person dedicated to managing the social accounts and ensuring prompt responses (ideally within 24 hours) to customer queries is important.
Partner network/ Indirect selling**:** Many developers follow their mainstream Influencers in their area to fuel their passion. This helps them to Learn — Unlearn — Relearn trends in the industry and grow as an individual. Identifying a handful of trusted partners with good networks, particularly in the developer’s segment, could be another great sales channel for Track.

C. Product / Feature Launch plan

The Product Launch plan is divided into 3 major parts. They are:

Pre Launch Phase
Launch Day
Post Launch Phase

● Pre Launch Phase:

Identify a date of launch giving sufficient time for planning and product/launch readiness. This will require working with different teams, including product, engineering, sales, marketing, and support.
Identify a handful of beta users, and gather initial data on the efficiency of using Track in their businesses, and assess how the usage of Track compares to some of the other collaboration tools that they may have used previously.
Refine the product messaging to reflect the focus on solving customer needs and adding business value.
Define a two-minute product pitch that can be used by teams across the company.
Conduct internal training sessions to ensure consistency in product messaging across various teams.
Having an FAQ document with product details is also very helpful.
Define KPIs to be tracked on the launch day for Track. These could be: a) Number of sign-ups, b) Unique visitors to the product page, c) Sign-ups for the various tiers, d) CAC
Building dashboards to track these KPIs would be the next step. It is important to ensure that the product branding (UX, color, language) works for the developer audience that we are targeting; e.g., the language should be simple.
Ready social media channels for the launch with the right messaging and gather initial traction and followers for the channels.

● Launch Day:

Publish a blog on the website and social channels announcing the launch of the product. This blog will also include quotes from users who participated in the beta test.
Have popular media channels (e.g., TechCrunch) cover the product launch and key features.
Run gamified audience engagement through early access, freebies, etc. on launch day.
Do live tracking of KPIs.
Effectively map out the customer journey and identify areas of friction during usage that cause users to drop off; this could be at the sign-up stage, the email confirmation stage, or the payment stage.
Reach out to customers who signed up, to understand their experience and gather feedback on areas of improvement. Repeat this for customers who did not sign up, to understand their reasons. This will help go back and make adjustments to the sign-up workflow.

● Post-Launch Phase:

Check all data, including:
Rates of acquisition, adoption, conversion, the efficiency of the various sales and marketing channels for Track;
Run analysis on the users signing up for the product to confirm that it matches the customer segment identified to make any changes if there is a mismatch;
Any areas of customer feedback that needs fixing, based on proof such as customer stories and testimonials from the initial adopters on the user experience and benefits of Track to their business.
Enhance ongoing customer support and long-term customer journey.

My Checklist;

Product — Market Fit — CHECK ✔️
Feature Optimisation — CHECK ✔️
ROADMAP/ BACKLOG — CHECK ✔️
Growth Experiments — CHECK ✔️
Go-To-Market Strategy- CHECK ✔️

Expected Growth; 🚀

Just looking back at numbers as to how we would grow by December 2021 after Testing and Implementing all the features to the Developer facing platform.