[GH-ISSUE #13] Stream FPS doesn't match preview's #12

Open
opened 2026-05-05 14:59:36 -06:00 by gitea-mirror · 15 comments
Owner

Originally created by @LordHDL on GitHub (Dec 1, 2013).
Original GitHub issue: https://github.com/zakk4223/CocoaSplit/issues/13

I set it to 60 fps. While streaming I can see very smooth video in both CamTwist and CocoaSplit's preview, but the resulting stream fluctuates between 55-60. My stream is stable, so pending output is always at 1 and dropped frames always at 0.

Originally created by @LordHDL on GitHub (Dec 1, 2013). Original GitHub issue: https://github.com/zakk4223/CocoaSplit/issues/13 I set it to 60 fps. While streaming I can see very smooth video in both CamTwist and CocoaSplit's preview, but the resulting stream fluctuates between 55-60. My stream is stable, so pending output is always at 1 and dropped frames always at 0.
Author
Owner

@zakk4223 commented on GitHub (Dec 10, 2013):

Are you seeing a mismatch between fps in/fps out or are they both showing the same less-than-60fps?

<!-- gh-comment-id:30237655 --> @zakk4223 commented on GitHub (Dec 10, 2013): Are you seeing a mismatch between fps in/fps out or are they both showing the same less-than-60fps?
Author
Owner

@LordHDL commented on GitHub (Dec 10, 2013):

They both show around 59.9 constantly. Only the decimal fluctuates so it might sometimes say 59.7 for instance. It never shows 58 or lower, only the stream itself is affected.

<!-- gh-comment-id:30237933 --> @LordHDL commented on GitHub (Dec 10, 2013): They both show around 59.9 constantly. Only the decimal fluctuates so it might sometimes say 59.7 for instance. It never shows 58 or lower, only the stream itself is affected.
Author
Owner

@zakk4223 commented on GitHub (Dec 13, 2013):

If you stream and analyze it with this: http://r-1.ch/analyze-twitch-vod.php what does it look like?

<!-- gh-comment-id:30498763 --> @zakk4223 commented on GitHub (Dec 13, 2013): If you stream and analyze it with this: http://r-1.ch/analyze-twitch-vod.php what does it look like?
Author
Owner

@LordHDL commented on GitHub (Dec 13, 2013):

stream analysis

I know stream technology is imperfect. Currently there is no 60 fps stream I've watched that is perfectly smooth, but back when I used different stream software I got a consistently better frame rate than what I'm getting now (basically as good as CocoaSplit's preview) using the same settings and CPU.

<!-- gh-comment-id:30508761 --> @LordHDL commented on GitHub (Dec 13, 2013): ![stream analysis](https://f.cloud.github.com/assets/5182602/1742295/484b0db4-63f8-11e3-9641-c942ea7e50fb.png) I know stream technology is imperfect. Currently there is no 60 fps stream I've watched that is perfectly smooth, but back when I used different stream software I got a consistently better frame rate than what I'm getting now (basically as good as CocoaSplit's preview) using the same settings and CPU.
Author
Owner

@LordHDL commented on GitHub (Dec 13, 2013):

So I decided to change some settings for testing, and with these new settings I did a live test.

live analysis

Huge improvement in frame variance. My guess is that changing from VBR to CBR did it, although in the past I used VBR with other software and didn't have this issue, which leads me to believe CocoaSplit is doing something funky with VBR.

<!-- gh-comment-id:30538281 --> @LordHDL commented on GitHub (Dec 13, 2013): So I decided to change some settings for testing, and with these new settings I did a live test. ![live analysis](https://f.cloud.github.com/assets/5182602/1745547/a49de9f6-642f-11e3-8391-383dff630453.png) Huge improvement in frame variance. My guess is that changing from VBR to CBR did it, although in the past I used VBR with other software and didn't have this issue, which leads me to believe CocoaSplit is doing something funky with VBR.
Author
Owner

@zakk4223 commented on GitHub (Dec 14, 2013):

The fact that the vod you analyzed produced no values for the x264-settings is also a bit odd, it certainly produced values for the vods I pointed it at. I dunno if that's some twitch quirk or an indication there was something odd with the encoder config though.

<!-- gh-comment-id:30589281 --> @zakk4223 commented on GitHub (Dec 14, 2013): The fact that the vod you analyzed produced no values for the x264-settings is also a bit odd, it certainly produced values for the vods I pointed it at. I dunno if that's some twitch quirk or an indication there was something odd with the encoder config though.
Author
Owner

@LordHDL commented on GitHub (Dec 14, 2013):

Yeah I found that weird too. I tried other broadcasters and browsers too to make sure and the same happened. The information seemed complete when I analyzed live streams, but for some reason now even that doesn't work. Is that analysis tool official or made by a 3rd-party?

<!-- gh-comment-id:30589576 --> @LordHDL commented on GitHub (Dec 14, 2013): Yeah I found that weird too. I tried other broadcasters and browsers too to make sure and the same happened. The information seemed complete when I analyzed live streams, but for some reason now even that doesn't work. Is that analysis tool official or made by a 3rd-party?
Author
Owner

@zakk4223 commented on GitHub (Dec 14, 2013):

It's a third party, and I bet the Twitch changes yesterday broke it

<!-- gh-comment-id:30589705 --> @zakk4223 commented on GitHub (Dec 14, 2013): It's a third party, and I bet the Twitch changes yesterday broke it
Author
Owner

@LordHDL commented on GitHub (Mar 6, 2014):

I streamed earlier today with the March 6 build and this issue is still present. I did a ton of testing beforehand using local recordings and the local videos don't exhibit the FPS drop. The Twitch analysis tool was updated a while back so I used it today.

stream analysis

Here's what this actually looks like on Twitch: http://www.twitch.tv/ihdl/c/3838233

As you can see it's hardly 60 fps smooth, yet it appears smooth in the preview, local recordings, and by extension CamTwist.

<!-- gh-comment-id:36935141 --> @LordHDL commented on GitHub (Mar 6, 2014): I streamed earlier today with the March 6 build and this issue is still present. I did a ton of testing beforehand using local recordings and the local videos don't exhibit the FPS drop. The Twitch analysis tool was updated a while back so I used it today. ![stream analysis](https://f.cloud.github.com/assets/5182602/2350396/0f2765fa-a570-11e3-9b91-1baf3d20aed2.png) Here's what this actually looks like on Twitch: http://www.twitch.tv/ihdl/c/3838233 As you can see it's hardly 60 fps smooth, yet it appears smooth in the preview, local recordings, and by extension CamTwist.
Author
Owner

@LordHDL commented on GitHub (Apr 17, 2014):

So apparently the preview option is contributing to this. I did some more testing and I consistently get those dips in frame rate you see in the image above when I have preview on. Turning it off resulted in a smoother output and a perfectly horizontal blue bar in the analysis. However, even doing this the live output still doesn't match the smoothness of the preview. It's just alleviated some.

This affects other frame rates too, like 30 fps, so best bet is to just leave preview off while live. There is still something else at work, though.

<!-- gh-comment-id:40721100 --> @LordHDL commented on GitHub (Apr 17, 2014): So apparently the preview option is contributing to this. I did some more testing and I consistently get those dips in frame rate you see in the image above when I have preview on. Turning it off resulted in a smoother output and a perfectly horizontal blue bar in the analysis. However, even doing this the live output still doesn't match the smoothness of the preview. It's just alleviated some. This affects other frame rates too, like 30 fps, so best bet is to just leave preview off while live. There is still something else at work, though.
Author
Owner

@LordHDL commented on GitHub (Apr 17, 2014):

Out of curiosity I did the analysis on your channel while you happened to be live. Here's what it gave me, followed by a test on my own channel a bit earlier:

zakklol

ihdl

<!-- gh-comment-id:40725392 --> @LordHDL commented on GitHub (Apr 17, 2014): Out of curiosity I did the analysis on your channel while you happened to be live. Here's what it gave me, followed by a test on my own channel a bit earlier: ![zakklol](https://cloud.githubusercontent.com/assets/5182602/2733132/cdeac868-c642-11e3-9b82-85374d078201.png) ![ihdl](https://cloud.githubusercontent.com/assets/5182602/2733137/e21b582a-c642-11e3-8066-f6feefd8ef9d.png)
Author
Owner

@zakk4223 commented on GitHub (Apr 18, 2014):

Hmm, I put in some debugging output locally and I think I know what is going on here:

If the entire 'capture frame->preview frame->encode frame' sequence takes longer than one frame (so 16ms at 60fps) the next frame is dropped. Since two threads can't update an OpenGL context simultaneously, there's a lock in the preview rendering code to prevent that from happening. So occasionally the main UI thread will grab the context and this then causes the CocoaSplit thread to block long enough that it will drop/delay a frame. You can make it worse if you grab the CocoaSplit window and move it around a lot, since it causes more UI redraws.

It was done this way because initially I tried to render the preview out to the screen as quickly as possible, because I was using it as a way to get fullscreen preview of BlackMagic devices. I've pretty much abandoned that use so I'm pretty sure I can rewrite this in a way that won't block the capture thread sometimes, but at a cost of increasing the latency of the preview sometimes.

Disabling preview should make things better, as you've noted.

I'm still not sure about the preview/live smoothness thing. I need to come up with a way to reproduce that locally. It may have something to do with the frame capture timing being slightly off from the 'native' framerate.

<!-- gh-comment-id:40835036 --> @zakk4223 commented on GitHub (Apr 18, 2014): Hmm, I put in some debugging output locally and I think I know what is going on here: If the entire 'capture frame->preview frame->encode frame' sequence takes longer than one frame (so 16ms at 60fps) the next frame is dropped. Since two threads can't update an OpenGL context simultaneously, there's a lock in the preview rendering code to prevent that from happening. So occasionally the main UI thread will grab the context and this then causes the CocoaSplit thread to block long enough that it will drop/delay a frame. You can make it worse if you grab the CocoaSplit window and move it around a lot, since it causes more UI redraws. It was done this way because initially I tried to render the preview out to the screen as quickly as possible, because I was using it as a way to get fullscreen preview of BlackMagic devices. I've pretty much abandoned that use so I'm pretty sure I can rewrite this in a way that won't block the capture thread sometimes, but at a cost of increasing the latency of the preview sometimes. Disabling preview should make things better, as you've noted. I'm still not sure about the preview/live smoothness thing. I need to come up with a way to reproduce that locally. It may have something to do with the frame capture timing being slightly off from the 'native' framerate.
Author
Owner

@LordHDL commented on GitHub (Sep 1, 2014):

Did a 60 fps stream recently and this issue is still present. Preview and local recordings are perfectly smooth, but with the same encoding settings a live stream performs at around 40-48 fps (the log says 60, though). Tested with the current stable build and not experimental CoreImage one.

<!-- gh-comment-id:54065575 --> @LordHDL commented on GitHub (Sep 1, 2014): Did a 60 fps stream recently and this issue is still present. Preview and local recordings are perfectly smooth, but with the same encoding settings a live stream performs at around 40-48 fps (the log says 60, though). Tested with the current stable build and not experimental CoreImage one.
Author
Owner

@zakk4223 commented on GitHub (Sep 2, 2014):

Are you watching the stream on a mac? I just noticed on a 2012 rMBP with safari twitch's video stats show 40-ish FPS meanwhile windows running on a 2010 mbp shows 60fps....and it's because on the mac the flash player is dropping frames constantly.

Does it look any different if you use live streamer (http://livestreamer.tanuki.se/en/latest/) to watch it via VLC?

<!-- gh-comment-id:54109987 --> @zakk4223 commented on GitHub (Sep 2, 2014): Are you watching the stream on a mac? I just noticed on a 2012 rMBP with safari twitch's video stats show 40-ish FPS meanwhile windows running on a 2010 mbp shows 60fps....and it's because on the mac the flash player is dropping frames constantly. Does it look any different if you use live streamer (http://livestreamer.tanuki.se/en/latest/) to watch it via VLC?
Author
Owner

@LordHDL commented on GitHub (Sep 2, 2014):

I am, however my experience is still varied so I don't think flash player is the only factor. Some observations:

  • I can watch other streamers on Twitch/Hitbox at 60 fps and see full smoothness in Safari on OS X that I can't achieve with CocoaSplit (except in preview/local recording).
  • I have some old Twitch videos (like the one linked above) that are almost full 60 fps with occasional drops (around 55 fps) and can see this reflected in iStat Menus GPU information.
  • I have a Hitbox video that appears to run at 40 fps in Safari and Chrome, but almost 60 when viewed with Livestreamer. A friend of mine (uses Windows) claims it did not look smooth while live but the archive itself looks smooth for him in browser even though it doesn't for me.

I certainly don't think flash is helping but there's clearly something else going on as this doesn't seem to affect users of OBS/XSplit.

<!-- gh-comment-id:54166896 --> @LordHDL commented on GitHub (Sep 2, 2014): I am, however my experience is still varied so I don't think flash player is the only factor. Some observations: - I can watch other streamers on Twitch/Hitbox at 60 fps and see full smoothness in Safari on OS X that I can't achieve with CocoaSplit (except in preview/local recording). - I have some old Twitch videos (like the one linked above) that are almost full 60 fps with occasional drops (around 55 fps) and can see this reflected in iStat Menus GPU information. - I have a Hitbox video that appears to run at 40 fps in Safari and Chrome, but almost 60 when viewed with Livestreamer. A friend of mine (uses Windows) claims it did not look smooth while live but the archive itself looks smooth for him in browser even though it doesn't for me. I certainly don't think flash is helping but there's clearly something else going on as this doesn't seem to affect users of OBS/XSplit.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/CocoaSplit#12
No description provided.