SharkWeek News Feed API

[mjstallard]

Connect News Feed API Specification

JSON API

Introduction

Word! I'd like to follow the JSON API spec in order to provide a nice standardized format. This means that, in theory, we can plug in a fancy library on each of the mobile platforms to make deserializing it a little easier (although this definitely a theory at this point). Regardless, it's a fairly nicely structured format.

There are some Python implementations of it at http://jsonapi.org/implementations/#server-libraries-python, which may make your life much easier :)

I'd like to suggest that we have unit tests around this, as it'll be a core part of the app and it'd be good to avoid regressions as the news feed definition evolves, if possible.

It should probably share the database with Sharknado, but probably shouldn't be colocated, so that we can scale it independently.

I know that @nymd had some ideas around server side tech (possibly using Tornado), but that is beyond my knowledge (and remit as a lowly iOS developer on this project) and hence is not in the scope of this spec :D

Example

Here's an example of how we use it already for Action Alerts from connect:

https://connect.berniesanders.com/api/action_alerts

{
   "data":[
      {
         "id":"139",
         "type":"action_alerts",
         "attributes":{
            "title":"Share Now - #AmericaTogether",
            "date":"Today!",
            "body":"When people stand together, and are prepared to fight back, there's nothing we can't accomplish. #AmericanTogether\n\n\u003cheader\u003e\r\n\u003ch2\u003e#AmericaTogether\u003c/h2\u003e\r\n\u003c/header\u003e\n\nOn Friday February 19th, we are rolling out another very big social media call to action on Twitter. The goal is to give Bernie a huge surge of online support, and to show as many on-the-fence voters that taking part in the democratic process is a powerful way to break through the hate-filled barriers that often divide so many of us. It’s all about millions of Americans (and thousands of Nevadans) coming together to bring about a political revolution, which is why we’ll be using the hashtag #AmericaTogether all day long.\n\nAll year long, Bernie has emphasized that this campaign is not about him — it’s about all of us. Which is why we believe the hashtag #AmericaTogether is a powerful embodiment of what this campaign is all about. It’s about uniting together over our similarities, and refusing to be divided by our differences, and we want to show the world that when we stand together, there is nothing that we cannot accomplish.\n\nIt would help us greatly if you could retweet our posts from  [@BernieSanders](https://www.twitter.com/berniesanders), and then adopt the hashtag as your own! Share your stories. What does it mean to you?\n\nPS – For greater effect, please be sure to download each of the translated posters below, and then upload them into your respective translated tweets!\n\n \n\nFor more social media sharable content to go:   [https://berniesanders.com/AmericaTogether/](https://berniesanders.com/AmericaTogether/)",
            "short_body":"When people stand together, and are prepared to fight back, there's nothing we can't accomplish. #AmericanTogether",
            "target_url":"https://www.facebook.com/media/set/?set=a.985805401474464.1073741843.124955570892789\u0026type=3",
            "twitter_url":"",
            "tweet_id":"700791753087934464"
         }
      },
      {
         "id":"112",
         "type":"action_alerts",
         "attributes":{
            "title":"Get Connect Alerts on your iPhone with Movement App",
            "date":"Download Now!",
            "body":"Movement, (just like the Connect site and many other Bernie tools, websites, and apps) was made by dedicated, hard working, volunteers who believe in Bernie's goals and want to help all volunteers get up-to-the-minute updates directly from the campaign on their mobile phones.  Download the app to stay connected on the go.\n\n**[Click Here to Download Movement from the iTunes App Store!](https://geo.itunes.apple.com/us/app/movement-elect-bernie-sanders/id1047784111?mt=8)**",
            "short_body":"\"Movement - Elect Bernie Sanders in 2016\" is now available in the app store for iPhones. This free app gives you access to sharable campaign messages, actions, news, and more.",
            "target_url":"",
            "twitter_url":"",
            "tweet_id":"695385140122419200"
         }
      }
   ],
   "links":{

   }
}

Endpoint definition

/newsfeed

The content that is included in a response by default is:

• News or Articles that are of article_type of PressRelease OR DemocracyDaily
• Videos

By default, the content should be English language. Later on, if we include Spanish content in the app, we should be able to pass a query parameter that defines the language to be returned.

Sort order

The current behavior on the iOS app is to fetch up to 30 News or Articles, and up to 5 Videos It might be nice to have some admin control over the mix of content (i.e., tweak the maximum number of each type), but that's not necessary for a first release.

The sort ordering (derived from the iOS app) is as follows:

• All news, articles and videos ordered by publish date, descending (i.e., most recent first).

It might also be nice to allow an admin to "pin" a news story, article or video to the top of the list, but again this is not necessary for a first release.

Required Fields

News and Articles

• title, e.g. "Improving the Rural Economy"
• body_markdown, e.g. "The Governator"
• excerpt, e.g. "I'll be back"
• timestamp_publish, e.g. "2015-12-29T04:23:16+00:00"
• url, e.g. "https://berniesanders.com/issues/improving-the-rural-economy/"
• image_url, e,g, "https://berniesanders.com/wp-content/uploads/2015/05/IMG_05471.jpg"
• type, e.g. "News" or "Article"
• id, e.g. "7fbb52b9-e747-4f7d-9b14-4b7ecdab4ac5"

Videos

• title, e.g. "Expand Social Security"
• video_id, e.g. "c4e9yiuKAmY"
• description, e.g. "Get to da choppa"
• timestamp_publish, e.g. "2016-01-13T19:08:43+00:00"
• thumbnail_url, e.g. "https://i.ytimg.com/vi/c4e9yiuKAmY/hqdefault.jpg"
• type, e.g. "Video"
• id, e.g. "7fbb52b9-e747-4f7d-9b14-4b7ecdab4ac5"

Sample response

Something akin to this should work:

   "data":[
      {
         "id":"7fbb52b9-e747-4f7d-9b14-4b7ecdab4ac5",
         "type":"video",
         "attributes":{
            "title":"Expand Social Security",
            "video_id_":"c4e9yiuKAmY",
            "description": "We should expand social security benefits, not cut them.",
            "timestamp_publish": "2016-01-13T19:08:43+00:00",
            "thumbnail_url": "https://i.ytimg.com/vi/c4e9yiuKAmY/hqdefault.jpg"
         }
      },
      {
         "id":"7de0c82f-2a63-4286-9b5c-84a0917f3a0f",
         "type":"news",
         "attributes":{
            "title":"Get Connect Alerts on your iPhone with Movement App",
            "body_markdown": "# DES MOINES, Iowa\nThere was fresh evidence on Sunday that confirms Bernie Sanders would be the most electable Democratic Party nominee for president because he performs much better than Hillary Clinton in matchups with leading Republicans in Iowa and New Hampshire, two key general election battleground states.",
            "excerpt": "DES MOINES, Iowa – There was fresh evidence on Sunday that confirms Bernie Sanders would be the most electable Democratic Party nominee...",
            "timestamp_publish": "2016-01-10T00:00:00+00:00",
            "url": "https://berniesanders.com/press-release/electability-matters/",
            "image_url": "https://berniesanders.com/wp-content/uploads/2016/02/image1-1.jpg"
         }
      },
      {
         "id": "d75d6773-54ed-4767-8280-de7952d63b7b",
         "type":"article",
         "attributes":{
            "title":"I Endorse Bernie Sanders for President",
            "body_markdown": "#As prepared for delivery on February 5, 2016\nThank you. It’s good to be back in New Hampshire.",
            "excerpt": "Thank you. It’s good to be back in New Hampshire...",
            "timestamp_publish": "2016-02-05T00:00:00+00:00",
            "url": "https://berniesanders.com/i-endorse-bernie-sanders-for-president/",
            "image_url": "https://berniesanders.com/wp-content/uploads/2016/02/NAACP-President-and-CEO-Benjamin-Jealous-said-Sunday-that-black-Americans-are-doing-far-worse-than-when-Pres.-Obama-first-took-office-404x250.png"
         }
      }

   ],
   "links":{

   }

/newsfeed/:id

This will return a single record matching the id formatted as per the news feed above, or a 404.

Sample response

For example, a call to /newsfeed/7de0c82f-2a63-4286-9b5c-84a0917f3a0f would return:

{
   "data":{
      "id":"7de0c82f-2a63-4286-9b5c-84a0917f3a0f",
      "type":"news",
      "attributes":{
         "title":"Get Connect Alerts on your iPhone with Movement App",
        "body_markdown": "# DES MOINES, Iowa\nThere was fresh evidence on Sunday that confirms Bernie Sanders would be the most electable Democratic Party nominee for president because he performs much better than Hillary Clinton in matchups with leading Republicans in Iowa and New Hampshire, two key general election battleground states.",
        "excerpt": "DES MOINES, Iowa – There was fresh evidence on Sunday that confirms Bernie Sanders would be the most electable Democratic Party nominee...",
        "timestamp_publish": "2016-01-10T00:00:00+00:00",
        "url": "https://berniesanders.com/press-release/electability-matters/",
        "image_url": "https://berniesanders.com/wp-content/uploads/2016/02/image1-1.jpg",
      }
   }
}

[mjstallard]

Updated to no longer require pinning the most recent video to the top of the returned results

[dan-f]

Hey @mjstallard thanks for throwing this together!

How rigid are the API endpoints in your spec? I'm questioning the /newsfeed and /newsfeed/:id naming choice. I like the "plural nouns" and "concrete names" pattern suggested in the Apigee Rest API Conventions article. Perhaps it could look something like: /stories and /stories/:id (substitute "stories" for any better synonym).

Also, since we're supporting videos, news, and articles, do we need endpoints for each resource type? e.g. /videos // /videos/:id; /news-story // /news-story/:id; /articles // /articles/:id

If we were to use that pattern, then we might not even need the /newsfeed/:id item endpoint, since we could link directly to the corresponding videos or news or article endpoint.

[mjstallard]

@dan-f welcome to the party! :) 🎉

I hear you on the naming, for sure. I was scratching my head, and the best I could come up with was news feed (this was basically influenced by the iOS code, which consists of the NewsFeedController, showing value objects that implement a NewsFeedItem protocol)...but I'm totally open to other suggestions - /stories works for me, although it might be nice to find a noun that better encompasses videos...(insert the usual quote about naming and cache invalidation here! 😜 )

I see your point about an endpoint per resource type - I think this should be fine (and indeed more RESTful). We'll receive the type of the resource to fetch in push notifications to the mobile apps anyway, so this won't be a problem. Sounds good 👍

[jzoldak]

@mjstallard @dan-f @benmadrone

OK I've gotten the code up to this (revised) spec, as I understood it.
There are endpoints per resource type. I didn't rename "news" to another noun, as that's the DB table name and I thought it would be confusing if it were named otherwise (plus there didn't seem to be an obvious choice for a better noun).

One thing I was unsure of, and we can fix if it's wrong in a followup PR, is what do you do if the noun already has an 's' at the end? By convention, should it be /news or /newses? I went with /news.

[jzoldak]

Update - after chatting with @mjstallard we will get rid of the "news" "articles" and "videos" endpoints in favor of "/newsfeeds" which will return the items of all types.

This way the mobile clients on all platforms will get the same data, and not need any logic to combine them. Also for example, if we want to change the number of items or their relative balance, etc we can change all that logic on the server side rather than updating the apps and relying on users to upgrade.

Individual items will be retrieved by ID via this pattern: /news/123, /articles/123 or /videos/123.