http://www.xbox.com/en-US/live/engineeringblog/default.htm
A Look Inside Netflix on the Xbox 360
Posted January 22, 2010 by Shaheen Gandhi (SDE) - Visigoth
In this post, a few members of the Netflix client engineering team at Xbox bring you some details about how the application works.
|
|
|
|
|
We wanted to share with you some of the neater features of the latest release of our Netflix application and give some insight into how things work under the hood. Many of you provided a ton of feedback on the first version of Netflix, so we took that to heart while developing the next version as well as taking some of the cool work that Netflix did to further enhance the experience.
UI and Navigation
New IR Remote Functionality
Navigating around has always been a pretty straightforward affair on the console controller, but in Netflix 2.0, we added some functionality to the Xbox IR remote to make it even easier to use.
- You can page forward and backward through titles on your channels by pressing the right and left bumper on the controller. Now, the chapter skip buttons on the IR remote will perform the same action.
- When you’re looking at the details of an episode, you can use the left and right bumpers on the controller to jump to the previous and next available episodes. The chapter skip buttons work here too. This gives you a quick and easy way to find the episode you are looking for.
Episode Selection
When you go to the episode list for a series, we automatically select an episode for you based on certain criteria. This episode is also the one that will play if you use the quick-play option on the instant queue. We’ll first select the episode you watched previously; if you have completed that episode, the next one in the series will play.
Trick-play
In the lingua franca of video playback, trick-play is what happens when you fast-forward or rewind. You are presented with snapshot images of the content at set intervals (usually 10 seconds) so you can easily see where you are. While in trick-play mode, you can use the chapter skip buttons to move just one frame at a time.
Playback
To simplify bringing up the On-Screen Display (OSD), in Netflix 2.0 the functionality is tied to the A button on the controller and the display button on the IR remote. Using the OSD, you can change the display mode to fullscreen, stretch, native, letterbox, and party. The party display option is only available if you are in a Netflix party session.
Rich Presence
On Xbox LIVE, your friends can see what games you are playing through our rich presence system. Netflix can be told to either show the feature you are watching or to hide it. When allowing video rich presence, the title you are watching, and the episode name if it is a TV show, will be shown in rich presence, and box art for the feature will be displayed in the Friends channel on the dashboard.
You can change the Video Status setting whenever you want by bringing up the Xbox Guide, selecting the Settings blade and clicking Profile. From there, select the Edit Profile option followed by the Privacy Settings option. This setting is universal across Xbox video players that support rich video presence. If you chose to hide the videos you watch for one application, you chose to hide them for all applications.
Movie Channels
The channels you see in addition to the instant queue and party channels when you start Netflix now contain suggested movies that the Netflix service feels are in line with your watching/rating habits. Each of these channels can have up to 200 items giving you upwards of 800+ items available for instant watch without having to go to your PC. All items in these channels are navigable as if they are in your instant queue. You can play the content directly, or add items to your instant queue for future watching.
Party Mode
With this new release of the Netflix application, users are able to watch content together on the virtual couch. The application uses the LIVE party system that came about with the New Xbox Experience launched in November of 2008. Here's a tutorial on how to watch content with others.
Getting Set up
Starting a Netflix party is easy. Go to the Party channel and select “Start Party”. You will start an Xbox LIVE party automatically, and you will be shown a list of friends you can invite. The list is sorted with online friends at the top. You can also start a Netflix party by selecting a movie or television episode and clicking the “Watch with Party” option on the details page. Some items are not watchable in a party and have the Watch with Party and Suggest to Party buttons disabled. In order to watch a feature together, you must be in a LIVE party, so we automatically add you to one when you create or join a Netflix session. Each user is able to nominate a movie or TV episode for party play, but only the leader (identified by the star at his or her Avatar’s feet on the Party channel) can begin playing a feature. If parental controls are enabled on one of the users in the Netflix party, that user will have to enter the parental controls passcode if the feature is blocked by their ratings settings. Until the code is put in, the feature will not start for anyone.
Watching the movie
Once playback starts, the leader (identified by the star on the back of his or her chair) is the only one who can control playback of the feature. Anyone can use the Y button to bring up the emote wheel. Each emotion has 3 Avatar animations associated with it that randomly cycle as you select them.
It’s not proper to wear a hat while indoors, so we take the liberty of removing any you may be wearing. Don’t worry, you’ll get them back once you leave playback. The same goes for any Avatar props you may be carrying.
Streaming Video
Rebuffering
One of the major challenges of streaming video is adapting to changes in available bandwidth. In the first version of Netflix, we saw a number of users experience downgrades in their video quality, accompanied by the “Your Internet connection has slowed” message, shortly after starting playback. This was often due to an ISP boosting available bandwidth for brief periods of time in order to make individual downloads faster. Since we decide which quality level of the content your Internet connection can handle by measuring bandwidth at the very beginning of a download, this boost effect causes the client to believe there is more bandwidth than actually available. Accordingly, we picked a quality level whose bitrate was too high to sustain once the boost ended and the connection reverted to its typical bandwidth characteristics. This resulted in a rebuffering event, otherwise known as a “starvation.”
We’ve done a number of things in this release to reduce the frequency of starvations:
- We have switched to using Netflix’s new, more advanced VC-1 encodings when available. The VC-1 streams are much more efficient than their visually-comparable WMV9 counterparts. This allows us to better utilize available bandwidth by buffering more of the feature in less time.
- We have implemented something we are calling “Seamless Stream Switching.” In the past, a starvation meant having to wait while the app rebuffered a new stream. Now, we have the ability to detect a dip in bandwidth that would cause a starvation, and react before we actually starve. Instead of forcing you to wait while the speed test and buffering is redone, we can prepare the new stream behind the scenes while playing out the old one. See the next section for details on how it all works.
- Courtesy of Seamless Stream Switching, you can now switch up to a higher quality stream. Previously, you would always be locked in to the lower bitrate stream if your bandwidth had degraded during playback. Now, if your bandwidth improves sufficiently, your content bitrate could be upgraded as well.
(Mostly) Seamless Stream Switching
As mentioned above, we can now be proactive about switching to a new stream before a starvation occurs. Here’s some information on how the process works:
Like with the first version of Netflix, a quick speed test is done first to figure out what the Internet connection is capable of. Based on these results, a stream of an appropriate bitrate is chosen - one that should give you the highest video quality possible while still being likely to play back without interruptions (aka starvations).
Unlike the first version of Netflix, the client will continuously monitor the current streaming bandwidth and also the buffer level during playback. The buffer level is how much data has been read ahead from the network connection and buffered up in memory.
If the buffer level falls below a certain threshold–i.e., the buffer level falls into the starvation danger zone–then we start to get interested, algorithmically speaking. At this point, the client looks at several samples of bandwidth measurements to see how the Internet connection is performing, and we’ll also see if buffer levels have been declining. It might seem like a silly thing to do, but the reason we check the latter is that, directly after a seek or after playback starts, buffer levels could technically still be in the red zone; however, as long as they are still increasing, there should be no need for alarm. On the other hand, if buffer levels have been declining, and bandwidth measurements indicate that the client could be in danger of starving, then the client will initiate a switch downwards to a lower bitrate stream. More about that in a moment.
Now if, instead of falling, the buffer level actually rises above a certain threshold, then the client will look at several history samples of bandwidth measurements and see if the Internet connection really can take it to the next level. If so, then the client will initiate a switch up to a higher bitrate stream. Note: To avoid ping-ponging between higher and lower bitrate streams, a backoff period is enforced before switching up if there has been a previous switch down during the playback session. The backoff period increases substantially the more times a downward switch occurs. For example, suppose client starts out playing a piece of content at 2 quality bars, but then bandwidth drops and the client switches down to a lower bitrate stream. At this point, even if the connection improves to the point where the client could technically switch up again, the client will wait an additional period of time before allowing the switch to occur. If the client subsequently performs another switch down during that streaming session, then the backoff time before allowing an up-switch would be longer still. The basic idea is to avoid a cycle where fluctuations in connection bandwidth cause the client to continually vacillate between content quality levels.
Performing the Switch: The Seam
Once a stream switch is kicked off, several things happen. First, the client stops reading any data from the network for the old stream; it plays out whatever buffer has been accumulated for the old stream from here on out. Simultaneously, the newly-freed network resources immediately start grabbing data for the new stream. Once all the data from the old stream has been played, the client will hot-swap to the new stream.
Perhaps this is a good time to confess that it’s not completely accurate to call this process “Seamless Stream Switching.” As it turns out, there is a seam, but it’s very slight and consists of the screen below.
The graphic at the upper right shows you the quality level for the new stream you’ll be watching. You’ll see this screen very briefly (typically, for a second or less) during the switch over from old to new. During this time, the video pipeline is being reinitialized for the new stream. We’ve done as much of the work as we could ahead of time, but there are some things, like initializing the Direct3D device in accordance with the new video resolution, that need to happen right then. This process is fairly snappy but it’s not instantaneous, which is why this small delay exists.
Sometimes the client is unable to adequately buffer up the new stream before the buffer for the old stream runs out. This may happen, for example, if there was a sudden extreme drop in bandwidth or if the Internet connection is not capable of sustaining uninterrupted playback of the lowest bitrate stream. In these cases, you will unfortunately still have to wait to rebuffer while viewing the “Your Internet connection has slowed” starvation screen.
Streaming and Party Mode
The most important thing about the streaming experience in Party Mode is that the video stays in sync among people watching the content. We felt strongly that it would not be a great experience if your buddy laughed at a joke two seconds before you actually saw the funny part of the movie. Because we were optimizing for keeping everybody at the same place in the content, streaming in Party Mode behaves a little differently than streaming in solo mode:
- There is no Mostly Seamless Stream Switching in Party Mode. The seam during the switch, slight as it is, could be enough to throw synchronization off beyond the 500 millisecond threshold that we deemed our maximum allowed drift between participants.
- The headroom that we require for Party Mode playback is higher than for solo mode. For context, headroom is the amount of space required between your measured bandwidth and the selected content bitrate. In other words, for a Party Mode headroom value of say, 30% (hypothetically speaking), your measured bandwidth would need to be at least 1429 Kbps for us to select a 1000 Kbps content stream. This is Bandwidth * (1.0 – Headroom). Now, the reason why we require more headroom in Party Mode is because a starvation in Party Mode affects not only your user experience, but also that of everybody else in your party. When one person starves, everybody else in the party has to wait for that person to rebuffer to keep everyone in sync. We want to make sure that this is very unlikely to occur.
Inside Tips and Tricks for Tweaking Your Netflix Streaming Experience
Since you now know some details of how our streaming system works, there are a couple of things you can do to potentially make the system work better for you.
First off, if you absolutely hate Mostly Seamless Stream Switching, you can prevent us from using it by watching all your content in Party Mode. This works whether you watch by yourself or with others. All you have to do is start a Netflix party. If you choose to do this and don’t want to see your Avatar, then use the On-Screen display and use Display Mode to switch.
Secondly, if you have a spiky or otherwise unstable Internet connection, the Pause button is your best friend. Remember how I mentioned that the client will only consider switching down if 1) the buffer falls below a certain threshold, and 2) the buffer level had been decreasing over the last few seconds? Well, when playback is paused, we will continue to read data from the network, but the buffer can’t decrease since playback has been halted. Therefore, hitting the pause button right after playback starts and leaving the content paused for a short while (30 seconds, a minute or two, see what works best for you) is a cheap and easy way to force the buffer level to increase and potentially avoid a downward switch to a lower bitrate stream.
A nice look behind the scenes. Wonder what they have in store for Netflix 3.0. Search functions???
