Apple TV 4K vs Amazon Fire TV Cube Gen2 vs nVidia Shield TV 2019 vs Roku Ultra

[Manni]

I did a bit of research recently because I was looking for an external box primarily to stream 4K HDR content (especially Netflix and Amazon Prime) with Atmos and frame rate match. I have most channels in my UB900 but I wanted Atmos and nothing forced to the wrong refresh rate (everything is forced to the default rate with the UB900).

So I got in house an Apple TV 4K, an Amazon FireTV Cube Gen 2 and a Shield TV 2019.

[EDIT July 26 2020: I put all the info on setting up a Shield TV Pro 2019 in this post].

I didn't test a Roku Ultra 2019 because 1) it doesn't support Dolby Vision, which I'd like to have for testing purposes even if I don't need it as I have an Envy Extreme in the chain. And 2) because I got one two or three times over the last few years to test specific things and every single time the performance was horrible (stutter, etc) so it went back every time. I can't remember if I tested it using Wifi or Ethernet, but I didn't like it. It's probably an interesting option if you don't need HDR10+ or Dolby Vision. [EDIT: it also looks like the Roku Ultra doesn't support Atmos with Netflix, which rules it out for this comparison].

[EDIT: before suggesting another device for this comparison, please make sure that it supports Netflix and/or Amazon Prime in 4K HDR with Atmos, frame rate match and has an Ethernet port. HDR10+ or Dolby Vision is a plus, but not essential. We all know there are cheap sticks at $50 or less that have at least one of the above missing but are great if you don't care about these criteria. We have an Amazon FireTV stick in the family room and it's adequate for the price. I was looking for a streamer in the cinema room, so with the best possible picture and audio quality, primarily for Netflix and Amazon Prime, on a big screen.]

Long story short, the FTV Cube went back right away because its frame rate match feature only works in Prime, Netflix plays everything at the default refresh rate.

The Apple TV 4K also went back because it doesn't support 24.00Hz, 30.00Hz or 60.00Hz. Otherwise it's a great streamer, possibly the most refined of the three (if you can live with the remote). This bug has been widely documented in various places, but I wanted to post this comparison here to avoid dragging these discussions off-topic.

Unfortunately, the ATV forces all 24.00 content to 23.976 when match frame rate is enabled, which means that most recent Netflix Original content (and all upcoming Netflix Original content) is unwatchable (to me). I can't live with a dropped/repeated frame every 40 secs or so for The Irishman, Bird Box, Dark, etc...

The Shield has its own limitations, but with a 3rd party app called Refresh Rate (you don't need to sideload it, it's in the Google App store now) it displays Netflix content perfectly and switches to the correct refresh rate automatically. There is also a beta feature that does more or less the same but it's not automatic (you have to select the option in settings) and it sometimes get the refresh rate wrong. Sadly I couldn't get the Refresh Rate app or the beta rate match feature to work with Prime (it caused a content protection error), but I don't really care as my priority is Netflix.

[EDIT 28-04-20: The third party Refresh Rate app also allows the Shield to switch to the native resolution of the content, as long as you set the default to 4K (Netflix Never plays a title to a higher resolution than the default). This makes the Shield the only streamer to offer a "source direct" mode, which is ideal when using an external VP such as the Envy or the Radiance Pro.

Also, I was able to use the refresh rate app on the Shield to deal with Prime and other apps not compatible with the full auto mode that works with Netflix. I simply set it to display the refresh rate when the title start playing, stop the title bring up a quick menu assigned to a button on the remote (I chose volume+) to select manually the correct refresh rate, and restart playback. It only takes a few secs and there is no guesswork involved as the refresh rate displayed by the app is correct most of the time.]

Apparently it's possible to get both Prime and Netflix to work using the Kodi add-on, but the repositories I found were buggy, so I gave up.

I attach a screenshot of my comparison table. Let me know if you spot any mistakes. [EDIT: I added a row for the Roku Ultra 2019, though I haven't tested it due to its lack of Atmos support with Netflix].

Again my focus was streaming, especially Netflix. If you are a gamer or want to passthrough lossless audio (Atmos, DTS:X) with Plex or Kodi, the Shield is for you, and if you want the Apple ecosystem, the ATV is a no-brainer.

Had the ATV 4K not had this 24.00hz bug, I'd have kept it as I preferred it overall (we have iPhones and iPads in the house, so it does make more sense to us longer term). However, right now, the Shield fits my needs better, as my priority is perfect Netflix playback.

One of the reasons why I sent the ATV back is because a new version is expected by the end of the year. Apple said that they are aware of the frame rate match issue and might fix it in TVOS14 (it's still there in TVOS 13.4).

So the plan is to get a new model in October if they have fixed this frame rate bug (and ideally provide a better remote), mostly because the ATV supports a few channels I care about (such as NowTV in the UK and Apple TV+). I don't need it to play disc content, I have my Oppo 203 clone and my madVR HTPC for that.

Otherwise I'll move the Shield to the Kids room and will get a Shield Pro, as you need the USB ports to get IR capabilities (for iRule and my Denon universal remote) using a FLIRC.

I bought the first Shield in 2015 and sent it back almost right away, it was too buggy and incomplete.

I really like the new Shield, the new remote is great and it's the best remote of the lot (backlit, nice size, many hardware buttons, "where is my remote" feature with Alexa, etc). I hated the ATV remote (too small, frustrating touchpad, not enough buttons, not backlit, etc). The only thing it had for it was that you could charge it with a Lightning port. I can't remember the FTV cube remote, but it wasn't backlit either.

Kodi works well, even tough I don't plan to use it to play my movies. I especially recommend the Embuary skin (optional add-on skin). I use MyMovies with the HTPC (along with CMC) and if you generate Kodi metadata, Kodi on the Shield picks it up and the Embuary skin supports categories as Tags. I use them extensively, so it's great to have them here.

The FTV Cube is a good player for the price, especially if you can live with the wrong refresh rate in every single app except Prime. Otherwise, it's pointless.

I honestly don't care which player you buy or use, so please no fanboy flaming from any side. I just thought I'd share the research (especially the attached table) in case it's helpful to others. Also, if you don't see any problem with the ATV with 24.00 content, you're lucky, I envy you, but that won't help those of us who do see the issue and can't live with it.

I've only had the Shield for a day or so, I'll update the thread after I spent a bit more time with it if I have any new info to provide. [Edited above, the Shield is the only streamer with a source direct mode, so that settles the score if you have an external VP].

Hope this helps!

[AVSMike]

Thanks for the work. Yeah, all of them are flawed. I keep waiting for someone to get it right, but it does not seem to happen.

[David Vaughn]

I like the Roku's interface and it's speedy. Is it perfect? No, but I only paid $80 for it, so I'm not too concerned. 

[Manni]

I like the Roku's interface and it's speedy. Is it perfect? No, but I only paid $80 for it, so I'm not too concerned.

Hi Dave,

Does it do frame rate match with Atmos for at least Netflix and Prime? Does it not skip frames? Do you use it with ethernet?

As I said in the OP, I got one on a couple of occasions to test HLG and the performance (even without HLG) was horrible, lots of stutter, so it went back every time.

Also I decided to only test players with Dolby Vision, in case it ends up recycled in the family room at some point (and there is a new TV there!).

Which model do you have? I don’t really care about DV but I want perfect Netflix and Prime with Atmos and HDR if I test something else. At the moment, the Shield gives me perfect Netflix.

The Roku is supported by my Denon universal IR, and it has Apple TV+ and NowTV, so that would be a plus!

EDIT: I did some research and it looks like Roku doesn't support Atmos on its Netflix app (only on Vudu, Prime and Disney+), which crosses the Roku off the list. I'll happily get one in if someone confirms this has been fixed. However I've updated the title and table to include the Roku Ultra 2019.

[David Vaughn]

Manni,

I don't think it does Atmos with Netflix, but it's not a deal breaker for me in that regard. Most of my Netflix watching is TV shows, so Atmos is not a deal breaker. DSU seems to work out well on what I've watched. I do notice the occasional frame skip though, but try to ignore it  My Mom has an Apple TV and like you, I hate the remote. It's horrible. 

I looked at the Shield before buying the Roku, but read it had a lot of problems (this was 2 years ago). Someone will hopefully come out with "one box to rule them all!"

[Manni]

Hi guys, I’m not interested in any stick, especially those that don’t do Atmos, don’t have an ethernet port or don’t have match frame rate at least for Netflix (for both 23.976hz and 24.00hz content, as explained).

I already have a UB900 in the cinema room and a FireTV stick in the family room that do no Atmos at 60p, there are tons of cheap sticks that do a crap job at streaming content (or a great job for the price), please let’s keep this thread on-topic.

Please create a new thread to compare cheap sticks if that’s what you’re into, and only post here if you find one that fits the criteria I explained in the OP (please read the first post!) or have an on-topic question regarding the streamers tested for this comparison.

Thanks!

[claw]

I have tried a Roku Ultra, Chromecast Ultra, Fire TV Stick 4K, and ATV4K. 

The only one I immediately returned was the Roku; just too buggy. 

I used the Chromecast up and until I purchased my ATV4K. 

I purchased the Fire TV Stick so I could get Apple TV+ on my LG OLED that does not have a native Apple TV+ App available.  I didn't want to purchase a second relatively pricey ATV4K when there is a new model in the works.

Even with the 24 Hz bug, I stick with the ATV4K in my theater room.  Supposedly there will be a redesigned remote with the next Apple TV model. 

For me, any streaming device I use must support an Apple TV+ App which the Fire Stick and of course the Apple TV4K do.  There isn't a lot of content yet, but what is there is very good in my opinion.  Plus, if you purchased an iPhone or other Apple device after September 1 last year you qualify for a free year of Apple TV+.

Why did you list the ATV4K as having Movies Anywhere in Low Res?    Or is it just because Google movies are.    For the few movies I have purchased or redeemed in Movies Anywhere I can watch them in 4K and Atmos from either the Vudu App or iTunes.

[Manni]

Yes as I said I hope the ATV replacement will fix this and have a better remote, but we have only rumors for now.

Roku (any model) doesn't do Atmos with Netflix, so it's pointless. You confirm my impression of the various versions I tested over the last couple of years (didn't try the 2019).

If you don't mind the 24.00, 30.00 and 60.00 bug in the Apple TV, I agree it's the best streamer. In fact that's what I said in the first post.

Sadly, I can't stand stutter and I already have stuttery options, so the Shield fits the bill for now.

Re your question, the ATV doesn't support VP9 at all, and it doesn't support Movies Anywhere either. You have to log into Youtube to get it in your purchases, and although it upscales to 4K, it's actually 1080p content upscaled to 4K. Vudu app or iTunes only works in the US for this. You can't redeem your Movies Anywhere on these in the UK due to geo blocking. You are correct that this works for US users, but I did say that this was a UK-centric comparison, which is why you don't see Vudu in the table, as we don't have it.

I can access AppleTV on my FireTV stick or on my iPads, so that can wait.

I just refuse to pay £200 for a streamer that doesn't give me perfect Netflix and Amazon playback, because I already have imperfect playback.

The Shield isn't perfect but for £150 it gives me fully auto refresh rate for the apps fully compatible with the  3rd party app (including Netflix) and semi auto with the others (Prime for example): The app displays the refresh rate of the content, you select the correct rate in a popup (one key on the remote) and then play at the correct rate. I set my default to 50p in the UK, so almost everything is at the correct rate, except the odd show on Amazon Prime which I have to set manually. I'd rather live with that than dropped frames in Netflix with the ATV.

I wouldn't want to keep the Shield long term though, it's far less refined that the ATV, some channels are missing as I indicated (AppleTV, NowTV) and it's unstable. Today I couldn't get Google Play to download anything, I tried everything but had to restore the unit to factory defaults (the internet connection was fine otherwise). If it happens again, it's going back.

I really hope Apple will get it right by the end of the year. I can't believe that no one is able to deliver a streamer that does it right for everything. It's not that difficult, they just don't care.

[David Vaughn]

Yes as I said I hope the ATV replacement will fix this and have a better remote, but we have only rumors for now.

Roku (any model) doesn't do Atmos with Netflix, so it's pointless. You confirm my impression of the various versions I tested over the last couple of years (didn't try the 2019).

If you don't mind the 24.00, 30.00 and 60.00 bug in the Apple TV, I agree it's the best streamer. In fact that's what I said in the first post.

Sadly, I can't stand stutter and I already have stuttery options, so the Shield fills the bill for now.

Re your question, the ATV doesn't support VP9 at all, and it doesn't support Movies Anywhere either. You have to log into Youtube to get it in your purchases, and although it upscales to 4K, it's actually 1080p content upscaled to 4K. Vudu app or iTunes only works in the US for this. You can't redeem your Movies Anywhere on these in the UK due to geo blocking. You are correct that this works for US users, but I did say that this was a UK-centric comparison, which is why you don't see Vudu in the table, as we don't have it.

I can access AppleTV on my FireTV stick or on my iPads, so that can wait.

I just refuse to pay £200 for a streamer that doesn't give me perfect Netflix and Amazon playback, because I already have imperfect playback.

The Shield isn't perfect but for £150 it gives me fully auto refresh rate for the apps fully compatible with the  3rd party app (including Netflix) and semi auto with the others (Prime for example): The app displays the refresh rate of the content, you select the correct rate in a popup (one key on the remote) and then play at the correct rate. I set my default to 50p in the UK, so almost everything is at the correct rate, except the odd show on Amazon Prime which I have to set manually. I'd rather live with that than dropped frames in Netflix with the ATV.

I wouldn't want to keep the Shield long term though, it's far less refined that the ATV, some channels are missing as I indicated (AppleTV, NowTV) and it's unstable. Today I couldn't get Google Play to download anything, I tried everything but had to restore the unit to factory defaults (the internet connection was fine otherwise). If it happens again, it's going back.

I really hope Apple will get it right by the end of the year. I can't believe that no one is able to deliver a streamer that does it right for everything. It's not that difficult, they just don't care.

Manni,

I also think the low cost of these units is why no one gets it perfect for all of the apps that are available. I wouldn't think the profit margin is very large on these streaming products, right?

[Manni]

Manni,

I also think the low cost of these units is why no one gets it perfect for all of the apps that are available. I wouldn't think the profit margin is very large on these streaming products, right?

These units cost £150-200. That’s enough for a decent profit margin given the numbers involved.
Plus I don’t care paying more if they get it right.
This comparison includes the top streaming devices currently available, and they are considered too expensive and “high end” by many, compared to the £30-£50 for the entry-level models (a lot less on Black Friday).
The ATV is the closest, but unless they fix that bug (it’s nothing to do unless it’s a weird hardware limitation) they are not getting my money. At least not until the end of the year, if no one else has done better.
Anyway, I’m sharing this info because none of these are perfect, and users have to compromise. It took me a lot of time to do this research, so I’ve decided to share it for those who are looking at the best possible streamer. The table should help users decide what’s most important for them and what they are (un)happy to let go. My best compromise is probably not everyone’s.
if you are happy with what you have, stay happy

[David Vaughn]

These units cost £150-200. That’s enough for a decent profit margin given the numbers involved.
Plus I don’t care paying more if they get it right.
This comparison includes the top streaming devices currently available, and they are considered too expensive and “high end” by many, compared to the £30-£50 for the entry-level models (a lot less on Black Friday).
The ATV is the closest, but unless they fix that bug (it’s nothing to do unless it’s a weird hardware limitation) they are not getting my money. At least not until the end of the year, if no one else has done better.
Anyway, I’m sharing this info because none of these are perfect, and users have to compromise. It took me a lot of time to do this research, so I’ve decided to share it for those who are looking at the best possible streamer. The table should help users decide what’s most important for them and what they are (un)happy to let go. My best compromise is probably not everyone’s.
if you are happy with what you have, stay happy

Manni, I know exactly how much work this takes and I'm sure I'm not the only one that appreciates it. You're the best!

[Manni]

Certainly not the best, but the grumpiest, probably 

[AVSMike]

Certainly not the best, but the grumpiest, probably

You hitting grumpy old man early? "Get off my lawn"  

[madshi]

Thanks for the detailed comparison, Manni! 

It's so disappointing that none of these devices can do native resolution and only one can (reliably) do native frame rate.  

One thing I would love to see is a comparison of image quality. In theory it should be the same with all devices, after all all h264 decoders are supposed to output the exact same signal. However, sadly no source device I know supports 4:2:0 output for anything but 4K50/60 (although the HDMI & CTA spec allows 4:2:0 for any resolution and frame rate). So all these source devices almost always do chroma upscaling internally. Consequently it would be interesting to know which device uses which chroma upscaling algorithm (nearest neighbor vs bilinear vs bicubic). Also, I've been told that some source devices add banding artifacts. Potentially a source device could also screw up output levels or introduce other problems, e.g. non-defeatible noise reduction or sharpening or slightly shifted colors.

But I know, testing image quality is very difficult and time consuming, if possible at all.

One question: Do all of these devices *natively* support playing video files from USB or LAN, for the purpose of running test videos to judge image quality? I suppose it will be possible by using Plex or Kodi, but I'm not sure if those use the same playback chain as the device's native video payback engine, so running such test videos through Plex/Kodi might only tell us how Plex/Kodi look like, but not how the device's native playback engine looks like?

[Manni]

Thanks for the detailed comparison, Manni! 

It's so disappointing that none of these devices can do native resolution and only one can (reliably) do native frame rate. 

One thing I would love to see is a comparison of image quality. In theory it should be the same with all devices, after all all h264 decoders are supposed to output the exact same signal. However, sadly no source device I know supports 4:2:0 output for anything but 4K50/60 (although the HDMI & CTA spec allows 4:2:0 for any resolution and frame rate). So all these source devices almost always do chroma upscaling internally. Consequently it would be interesting to know which device uses which chroma upscaling algorithm (nearest neighbor vs bilinear vs bicubic). Also, I've been told that some source devices add banding artifacts. Potentially a source device could also screw up output levels or introduce other problems, e.g. non-defeatible noise reduction or sharpening or slightly shifted colors.

But I know, testing image quality is very difficult and time consuming, if possible at all.

One question: Do all of these devices *natively* support playing video files from USB or LAN, for the purpose of running test videos to judge image quality? I suppose it will be possible by using Plex or Kodi, but I'm not sure if those use the same playback chain as the device's native video payback engine, so running such test videos through Plex/Kodi might only tell us how Plex/Kodi look like, but not how the device's native playback engine looks like?

Hi there!

In fact none of them does the correct frame rate with everything, the Roku had many bugs and a lot of stutter when I tested it. Maybe the 2019 version is better, but I only put it in green because I had not tested it and didn't want to make an assumption. In theory, it has frame rate match for everything. But then so does the ATV, and it clearly isn't the case when you look more closely.

Re your question, they don't have the same PQ but you have to make a distinction between media players and streamers, although these can be seen as both.

For me, the above are primarily streaming boxes, except maybe the Shield. While I expect them to deliver the correct frame rate when they advertise the feature (as do the ATV 4K, the FTV Cube, and the Roku), the PQ is in essence compromised because it's streaming. So you get a lot of compression issues, occasional stutter, all the reasons why I wouldn't use any of these boxes to play critical content. I didn't compare PQ in detail because of that, so they could very well have banding issues, but frankly that's nothing compared to the streaming artifacts that you get anyway because of the limited bandwidth. Also none of them (except the shield) are able to passthrough lossless audio, so that disqualifies them immediately as high-end mediaplayers. They only play Atmos MAT (low bandwidth for streaming) and not Atmos or DTS:X present in 1:1 rips from bluray or UHD Bluray.

The Shield is designed as a more versatile player, and it's the only one that is able to play 1:1 rips in full quality for audio and video. However, I didn't look into it in detail because I use my HTPC (with madVR ) or my Oppo 203 clone to play any critical content. I have zero interest in looking at PQ for mediaplayers, they all have issues with banding, stutter, etc. This is why I went for the Oppo with MyMovies as a front end. It's the best player because it's an actual standalone player, and not a mediaplayer emulating one. The HTPC is still the best source though, because the front end is better and madVR does all the chroma upscaling.

Re quality, as you say and as we have discussed many times, most consumer sources only send 4:2:0 at 50/60p, and 4:2:2 (or higher) otherwise. This is not really an issue with these streamers, because the streaming quality is the limiting factor. Anyone serious about ultimate PQ wouldn't use any of these for local/network playback anyway.

At first sight, the PQ of the Shield when playing 1:1 local files with Kodi looked excellent, but not as good as the Oppo or the HTPC. I didn't look into it in detail because I don't plan to use it as a primary source that way, but I'll comment later if I can put my finger on the reason.

The biggest downside for all of these is the lack of source direct, which is a far bigger issue than the chroma 4:2:2 vs 4:2:0, because it prevents an external VP such as the Envy or the Radiance Pro from doing the luma upscaling.

Arguably, the impact of the quality of the chroma upscaling is minimal with UHD content as the chroma layer is 1080p, and it gets more and more relevant as the resolution goes down. But when upscaling DVD or bluray content, the lack of source direct becomes a lot more visible.

For this reason, they will always be limited from a PQ point of view, compared to an Oppo 203 (or a Zapitti Pro) that offer a proper source direct mode and are able to output the native resolution of the content, even if chroma upscaling has to be 4:2:2 for non 50/60p content. The Dune HD 4K Pro can also do source direct but it has banding and/or frame rate issues.

However, the Shield has an AI Upscaling feature that is quite impressive for lower res content, up to 30p. It also has a demo mode that you should look at, because you can not only display the content side by side with it enabled / disabled, but you can also move the separation bar between the two pictures to choose which part of the picture has it enabled or disabled. I haven't had the time to look at this in detail, but I reckon that the Shield and the ATV would have the best upscaling overall (the ATV is very good, the Shield might be a bit better). Nothing at madVR/Envy level, of course, but "good enough" for compromised streaming.

All these devices support Plex (I think), but I didn't test local playback with that. I only tested Kodi for casual playback on the Shield, and it worked well. You'll have to look at the table to see which ones support USB or MicroSD. The ATV doesn't support any external storage.