I’d like to approach this problem from two different user group. One is business user, i.e., user
who use FB for advertising; the other is regular non-business user. Reason is these two groups
use different suites of FB product and have different purpose for using fb. I’m going to talk
about these in details later on. !
Let’s start with regular non-business user. The majority of the fb app I’m familiar with actually
target this user segment. For this segment, I’m assuming the goal of the improvement is to
increase user engagement. Do you think this is fair? I’m going to define a more specific metric
regarding engagement as we start to talk about the specific improvement. For now, we just
agreed on the high level goal. !
In order to find improvement in existing product, I would propose to look at current user
behavior/activity. My hypothesis here is that if user is currently doing something frequently,
but it requires them to take multiple steps, then if we could simplify this for them and make it
easier to use, we would expect them to buy in. So let’s take a look at what users are currently
doing. There’s a couple of ways for us to perform this analysis, we could look at user activity
log (i assume we have this kind of granular data) or we could analyze message (NLP, text
analytics). !
Let’s say we look at user activity log and noticed that when user use the “Place” feature, they
often look for new restaurants nearby. But currently, in order to do that, they need to go
through multiple steps. (Go to places ->restaurant). So based on this fact, we could improve fb
by adding a new notification that recommends new and trending restaurants nearby for user.
Based on my earlier hypothesis, I would expect user to find this feature useful and increase
usage of Places. To measure the effectiveness/success of this feature, I will define an
engagement metric as weekly # check-ins or page views/#DAU. **Note that for DAU, I’m only
considering users who have at least used Places feature before. I will also track the proportion
of users who have check in at least once/ #DAU on fb to measure the user base. **!
The reason I choose weekly is user probably won’t need to visit that page on a daily basis. I’m
also going to track the weekly # of posts/comments /#DAU to make sure we are not impacting
the existing feature. !
In order to test this feature, Note that if user start to use this more and go to more events, it is
possible that their friend may go to more events as well (network effect). So I would random by
market/cluster (comparable, i.e, one market does not have high usage of check in initially won’t
work). I would implement the change for the test group, for the control group, I would replace
the module with previous regular newsfeed. The timeframe of the test is determined by the
minimal sample requirement. I would run it for at least 2 weeks and see if there’s any significant
difference between the two groups. I would pay attention to novelty effect and make sure the
test is run for at least 2 weeks. Also pay attention to overall FB statistics because we don’t
want adverse impact on other product. !
How to improve ads:
I would assume the goal here is to increase the conversion rate. The most intuitive approach
here is to distribute ads to users based on their location. The hypothesis is people are more
likely to buy/visit from local business.!
For business with multiple location, allow them to use information from each of their pages to
add dynamic ad copy, so each ad is lcoalized for its corresponding store. The call-to-action
button is also location based. i.e., when user hit get location, the navigation is to their local
store. !
Better target people See the percentage of people nearby who have seen their ad. !