FFmpeg lib + UDP + Qt issuesHello,
First of all, thank you to this amazing community of developers! I have
received a tremendous amount of help both directly and indirectly thanks to
you all.
I am in need of some help regarding reading encoded video udp packets and
decoding them into a Qt display. I have managed to read the video packets
and display them in Qt but video playback 'stutters'. Checking the FPS of
frames transmitted via the FFMpeg library to Qt, I get ~30 frames per
second, so I'm led to believe that FFMpeg itself is sending me the same
frames multiple times a second. Here are the steps I took:
1. Broadcast an H.264 encoded video stream via VLC to
udp:/238.38.38.38:1234
2. Use FFMpeg lib to decode the stream. Here is my function (apologies
in advance, if there is a better way to display code, please let me know!):
bool QVideoDecoder::decodeSeekFrame(int after)
{
bool done=false;
while(!done)
{
// Read a frame
timeoutTimer.start();
if(av_read_frame(pFormatCtx, &packet)<0)
return false; // Frame read failed (e.g. end of
stream)
timeoutTimer.stop();
if(packet.stream_index==videoStream)
{
int frameFinished;
avcodec_decode_video2(pCodecCtx,pFrame,&frameFinished,&packet);
// Did we get a video frame?
if(frameFinished)
{
ffmpeg::AVRational millisecondbase = {1, 1000};
int f = packet.dts;
int t =
ffmpeg::av_rescale_q(packet.dts,pFormatCtx->streams[videoStream]->time_base,millisecondbase);
if(LastFrameOk==false)
{
LastFrameOk=true;
LastLastFrameTime=LastFrameTime=t;
LastLastFrameNumber=LastFrameNumber=f;
}
else
{
// If we decoded 2 frames in a row, the last times are okay
LastLastFrameTime = LastFrameTime;
LastLastFrameNumber = LastFrameNumber;
LastFrameTime=t;
LastFrameNumber=f;
}
// Is this frame the desired frame?
if(after==-1 || LastFrameNumber>=after)
{
// Convert the image format (init the context the first time)
int w = pCodecCtx->width;
int h = pCodecCtx->height;
img_convert_ctx = ffmpeg::sws_getCachedContext(img_convert_ctx,w, h,
pCodecCtx->pix_fmt, w, h, ffmpeg::PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL,
NULL);
if(img_convert_ctx == NULL)
{
printf("Cannot initialize the conversion context!\n");
return false;
}
ffmpeg::sws_scale(img_convert_ctx, pFrame->data, pFrame->linesize, 0,
pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize);
// Convert the frame to QImage
LastFrame=QImage(w,h,QImage::Format_RGB888);
for(int y=0;y<h;y++)
memcpy(LastFrame.scanLine(y),pFrameRGB->data[0]+y*pFrameRGB->linesize[0],w*3);
// Set the time
DesiredFrameTime =
ffmpeg::av_rescale_q(after,pFormatCtx->streams[videoStream]->time_base,millisecondbase);
LastFrameOk=true;
done = true;
}
}
}
av_free_packet(&packet);
}
return done;
}
I have a periodic timer that requests the latest frame from FFMpeg. If I set
the timer to a low rate (20 FPS), I get the video played back but at a slow
rate. If I set the timer to a high rate (60 FPS), I get the frames as soon
as they are ready, but the video stutters. I figured that I should request
the frames back as fast as possible.. Any insight on how to avoid the
stuttering is appreciated.
3. Send the QImage back to Qt to be displayed -- I have tested the
frames received by my Qt callback per second and can confirm that I am
receiving ~ 30 FPS which leads me to believe the stuttering is caused by
FFMpeg lib.
4. I also wrote an interrupting callback to be called by FFMpeg if any
of the blocking functions (i.e. av_read_frame) took too long:
Timer timeoutTimer;
static int decode_interrupt_cb(void)
{
if (timeoutTimer.getElapsedTimeInMilliSec() > 1000)
{
qDebug() << "Video Timed Out!";
return 1;
}
return 0;
}
Although I sometimes do receive the "Video Timed Out!" call, the
av_read_frame function still blocks...
Any help would be greatly appreciated!
阅读(1694) | 评论(0) | 转发(1) |
