Fetching Patreon Data

Patreon is a delight to scrape. Actually, scrapping is the wrong word for it – the frontend of Patreon is a react application that calls a number of very sensibly designed json end points. Call the same endpoints and you get delightfully clean json that exactly matches what gets displayed on the site.

A disclaimer – this is undocumented as far as I can tell – the publicly documented API (JS Implementation) is targeted at creators and provides access to privately information only visible to creators. All of this could change at any point.

I was all ready to parse HTML, but looking at the source there was a beautiful JS object containing all the data needed to display most pages.

"data": {
  "attributes": {
    "created_at": "2016-04-30T13:58:22+00:00",
    "creation_name": "Entertainment",
    "display_patron_goals": false,
    "earnings_visibility": null,
...

Even better, at the tail end of the long object is the call to fetch just the JSON:

"links": {
  "self": "https://api.patreon.com/campaigns/355645"
}

So as long as I can get the ID of a campaign, I can get all the information about it in an easily processed format. Thanks to the explore pages and a bit of network monitoring reveals calls to the following URL’s:

https://api.patreon.com/explore/category/12?include=creator.null&fields[user]=full_name,image_url,url&fields[campaign]=creation_name,patron_count,pledge_sum,is_monthly,earnings_visibility&page[count]=20&json-api-version=1.0';

Inspection of the different category pages reveals the number after ‘/category/’ runs from 1 through 14, and 99 for the ‘All’ category. This way, I can fetch all the top campaigns then use the campaign API to retrieve detailed information.

An interesting note – the data structures reveal a lot about how site has been designed and where complexity can be added later – multiple campaigns per user, links between campaigns, etc.

Full code for my scrapper is after the break – I’ll be diving into analysis next.
Continue reading “Fetching Patreon Data”

Quick Analysis – Patreon Funding

Inspired by the launch yesterday of the Patreon funding campaign for Movies with Mikey, a movie analysis YouTube channel, I’ve performed some rudimentary analysis of how Patreon donors fit into donation tiers.

Movies with Mikey Patreon

A quick primer – Patreon allows creators to collect donations from supporters on an ongoing basis as opposed to a one-time engagement as with Kickstarter. Donations can be by month or by produced work. While various donation levels provide perks or recognition, Patreon tends to be more focused on “support” than perk compared to other funding platforms. As a result, tiers tend to be less about “value for money” and therefore more interesting to analyze.

Patreon, as with Twitch, limits the information publicly available to summary information, but still enough to assess donation breakdown given some reasonable assumptions. For Movies with Mikey (MWM), we are given the total donations and how those are broken down by tier.

While we would like to know the specific distribution of donations, we can estimate the average donation per tier to get an idea of how donations break down. One caveat is that there is some missing data – the sum of donors in tiers only adds up to 744, so 39 supporters or about 5% didn’t select a tier and we have no idea where they fall. To analyze the breakdown, I made a simple spreadsheet to allow easy estimation of donation amounts.

This estimation underestimates the total by 1.8%, which given the ‘missing’ donations suggests a fair degree of accuracy. There are a few hypothesis that come out of this:

  • Donors give the minimum to be in a given tier.
  • There are likely a few high-end donations that pull the average of the top tier up.

I’d stress that this is exploratory work at the moment that uncovered some reasonable hypotheses. To confirm them and to make any recommendations will entail seeing if this pattern holds for other Patreon campaigns which I hope to do in the next few days looking across categories and sizes of campaigns.

One last comment I also plan of returning to is the nature of sites that provide summary statistics. By limiting the information available, they are creating a market for 3rd party scrapers and losing control over the data. For Patreon, there is Graphtreon and Kickstarter has Kicktraq. I’m torn on how the platforms and their users should feel about these, particularly as personal finances are often involved, but I’ll delve more into the issue later.

Amazon’s Twitch Prime Move – How can we see the result?

When Amazon made their move to create Twitch Prime (in what I called the best marketing move of all time), they had all the data – cross usage rates, demographics, subscription rates, etc. Now as we look from the outside, particularly as we’ve developed a number of alternate hypothesis, can we see and evaluate the success of the program?

As outsiders, while we don’t have direct access to the data, there are a number of point we can observe.

Market Embrace

Amazon is leveraging market forces, expecting that the benefits to streamers and viewers are enough to drive adoption. We can watch what streamers do to see if they are seeing the benefits. If Twitch Prime is creating value for them in the form of incremental subscribers, successful streamers should be heavily promoting it to their viewers.

We can certainly see some adoption today in on-stream pitches, stream titles, channel pages and social media mentions.

 

At the moment, I would hypothesize that there is a lot of excitement over Twitch Prime so current promotions aren’t necessarily market tested. The real proof will be in a month or two to see how normalized the messaging becomes.

 

We can also watch what Amazon does. They may opt to drop or modify the program if it isn’t delivering the results they are looking for.

Increased Viewers

It is a few steps away, but we could predict that because of Twitch Prime, more people participate on the platform, driving incremental views. Similarly, if viewers subscribe to more channels, they are more likely to regularly watch more, boosting total hours watched and views.

Total views and current viewers are the only numbers available, but watching them over time may provide some insight into overall platform growth thought there are many non-Prime factors impacting these metrics.

Watching Chat Participants

Twitch provides a chat interface where viewers interact with each-other and the streamer. A couple of features of the chat makes it worth watching. Users have “badges” that indicate if they are Prime Members, Subscribers, Moderators and a number of statuses. When viewers subscribe, a message is also sent to the chat.

Twitch Chat showing badges and subscription messages.
Twitch Chat showing badges and subscription messages.

What to look at in the example –

  • Sirlexon – the ‘POW’ is the badge that indicates subscribers while the crown indicates Prime members
  • You can see that Bline_Ophtalmologist subscribed using Twitch Prime
  • EQWashu has a sword that indicates they are a moderator
  • Franch1s3 has no badges

What this means is that we can test our hypothesis by counting users in the twitch chat to see how many users are subscribers and/or Prime members.

Twitch Prime: Alternate Hypothesis

I proposed yesterday that Twitch Prime is an amazing move by Amazon to help turn streamers into salespeople for Amazon Prime. Clearly, people love the new program and perks – it is showing up all over twitch in channels I follow. For example –

Twitch Prime Sign-up banner for The Attack
Twitch Prime Sign-up banner for The Attack

If you want to hear 10 minutes of Amazon love, here’s Day9 and friends discussing Twitch Prime and how great Amazon Prime is (beyond just for the twitch offerings…). It also touches on the oh-so-important community parts of twitch and subscriptions.

Even when streamers aren’t giving direct calls to action to join Amazon Prime, they can’t resist discussing the other positives to Prime. Day9 is a big enough deal in the streaming world that he’s heard from many other subscribers and raises a couple of interesting points based on what he’d heard –

  • Some streamers reported their subscribers quadrupling over the weekend when TwitchPrime was announced.
  • Instead of hanging out watching for 6 months before becoming a subscriber, with Prime they’re more likely to dive right in a to test the community waters.
  • The first subscription is the highest hurdle – so the motivation may be to drive those incremental subscriptions as people are more likely to make a subsequent full-cost subscription.

That last point could serve as an alternate (or additional) motivation for Twitch Prime. Tomorrow, I’ll explore how these hypothesis could be tested.

The Marketing Genius of Twitch Prime

Amazon rolled out Twitch Prime at this year’s TwitchCon and I’m convinced that it is the most genius marketing move of all time. Lets walk through it.

Amazon bought Twitch, a video game streaming site for ~$970M in 2014. At the time, the purchase was justified as buying the platform and audience to help it compete in the future of online video consumption against Netflix, Hulu and more importantly, YouTube. At the time of the sale, Twitch had 55 million users, accounted for over 40% of all online  live video, and had users spending over 100 minutes per visit on the site.

Video streamers on twitch become celebrities and create communities with their viewers. Streamers get paid in a number of ways, but the most important is that fans can subscribe to their channel, typically for $5/month. There are usually some associated perks, but the main draw is to support the streamer and to be part of the community.

Needless to say, streamers have gotten extremely good at attracting subscribers with sub goals, calling subscribers out by name , and pushing the community aspects. It is hard to understate the strength of the communities that are created out of this system and the loyalty of subscribers – NPR should be jealous.

Into this comes Twitch Prime, where Amazon Prime members receive a number of Twitch-specific perks. The most important is that prime members receive one free channel subscription. Streamers get paid the same as for a regular subscription but prime members don’t pay anything.

Excited yet?

The immediate effect is that all existing prime subscribers can have a lot of subscriptions that they can “give” to steamers, and there is some evidence of that happening –

2016-10-05-08_08_45-jp-mcdaniel-on-twitter_-_so-awesome-to-see-how-many-new-subs-on-twitch-thanks-to

Looking further out, the impact will go well beyond people connecting their Amazon and Twitch accounts.

Twitch Streamers are now Amazon Prime’s sales force.

Why just subscribe to a channel when you can sign up for Prime and get a ton of ever-expanding perks? Streamers are already adjusting –

2016-10-05-08_10_06-top-channels-twitch 2016-10-05-08_13_01-top-channels-twitch 2016-10-05-08_14_07-top-channels-twitch

The strength of this move is staggering – it takes a functional, fairly corporate product and makes it a badge of community for an attractive demographic resistant to other forms of marketing. It is every marketer’s goal to turn customers into advocates, but Amazon has, by making a small incentive tweak, introduced a whole new platform strategy that feels like a win to everyone involved.

HowTo Get Your PMP: Overview

Five years ago I received my Project Management Professional (PMP) certification from PMI. In the time since, I’ve regularly given advice to peers on how to get certified and what it takes to keep current. Over the next four posts I’ll be covering what worked for me, updating this post to serve as a table of contents.

DISCLAIMER: This is what worked for me – everyone learns differently and PMI keeps evolving their requirements and process.

  • Counting Up Work Experience
  • Satisfying The Training Requirement
  • Passing The Test
  • Continuing Education

Here’s the short version for the impatient –

  • Don’t read the PMBOK for your primary source of studying – it is meant as a reference and doesn’t work as a textbook. Read a study guide, such as Rita Mulcahy’s PMP Exam Prep instead.
  • Been working over 5 years in a role slightly beyond an individual contributor? You have the experience needed without needing to have had the title of “Project Manager”.
  • Get as many practice tests as possible and do them all.
  • Working as a PM? You’ll only need a few PMI webinars to satisfy your CEU’s.

Daily Reads 10/3

Daily reads for 10/2/2013

Too many articles on the Silk Road bust, but it is fascinating…

Reads for the Week 9/9 – 9/14 (part 1)

After a week where I had some trouble finding the time to do daily posts, I’ve automated part of the process so hopefully next week these will be more regular. This part 1 of what I read this week is mainly the gaming and computing news I’ve read. Articles are roughly in order of interest for easy skimming. Expect part 2 later today with more general articles.

Reads for 9/5 & 9/6

Life got in the way of updating, so here’s a slightly longer list of readings from the past several days. In no particular order…

Recipe of the day: Green Onion Manicotti

More Than 800,000 Scientific Papers In One Beautiful Infographic