Subject: bug#23256: vlc not building after update to ffmpeg 3.0

From: Christopher Allan Webber <[email-masked]>
Date: Sat, 09 Apr 2016 14:21:39 -0500

vlc is failing to build after the update recently to ffmpeg 3.0.  I
verified that it builds prior to this change.

Should we include an older version of ffmpeg to build vlc with for now?
Or should we try to figure out what to do to to get vlc to build with
ffmpeg 3.0?  Maybe for now we should do the former for the aim of the latter.

Here's the output while attempting to build.

  CC       codec/libvorbis_plugin_la-vorbis.lo
  CC       codec/avcodec/libavcodec_plugin_la-video.lo
codec/avcodec/video.c: In function ‘InitVideoDec’:
codec/avcodec/video.c:226:5: error: implicit declaration of function ‘avcodec_alloc_frame’ [-Werror=implicit-function-declaration]
     p_sys->p_ff_pic = avcodec_alloc_frame();
     ^
codec/avcodec/video.c:226:21: warning: assignment makes pointer from integer without a cast
     p_sys->p_ff_pic = avcodec_alloc_frame();
                     ^
codec/avcodec/video.c:308:34: warning: assignment from incompatible pointer type
     p_sys->p_context->get_format = ffmpeg_GetFormat;
                                  ^
codec/avcodec/video.c:438:9: error: implicit declaration of function ‘avcodec_free_frame’ [-Werror=implicit-function-declaration]
         avcodec_free_frame( &p_sys->p_ff_pic );
         ^
codec/avcodec/video.c: In function ‘lavc_dr_GetFrame’:
codec/avcodec/video.c:996:25: error: ‘PIX_FMT_PAL8’ undeclared (first use in this function)
     if (ctx->pix_fmt == PIX_FMT_PAL8)
                         ^
codec/avcodec/video.c:996:25: note: each undeclared identifier is reported only once for each function it appears in
codec/avcodec/video.c: At top level:
codec/avcodec/video.c:1305:25: error: return type is an incomplete type
 static enum PixelFormat ffmpeg_GetFormat( AVCodecContext *p_context,
                         ^
codec/avcodec/video.c:1305:25: error: conflicting types for ‘ffmpeg_GetFormat’
codec/avcodec/video.c:111:25: note: previous declaration of ‘ffmpeg_GetFormat’ was here
 static enum PixelFormat ffmpeg_GetFormat( AVCodecContext *,
                         ^
codec/avcodec/video.c: In function ‘ffmpeg_GetFormat’:
codec/avcodec/video.c:1317:5: error: invalid use of undefined type ‘enum PixelFormat’
     for( size_t i = 0; pi_fmt[i] != PIX_FMT_NONE; i++ )
     ^
codec/avcodec/video.c:1317:30: error: dereferencing pointer to incomplete type
     for( size_t i = 0; pi_fmt[i] != PIX_FMT_NONE; i++ )
                              ^
codec/avcodec/video.c:1317:37: error: ‘PIX_FMT_NONE’ undeclared (first use in this function)
     for( size_t i = 0; pi_fmt[i] != PIX_FMT_NONE; i++ )
                                     ^
codec/avcodec/video.c:1319:9: error: invalid use of undefined type ‘enum PixelFormat’
         const AVPixFmtDescriptor *dsc = av_pix_fmt_desc_get(pi_fmt[i]);
         ^
codec/avcodec/video.c:1319:67: error: dereferencing pointer to incomplete type
         const AVPixFmtDescriptor *dsc = av_pix_fmt_desc_get(pi_fmt[i]);
                                                                   ^
codec/avcodec/video.c:1324:9: error: invalid use of undefined type ‘enum PixelFormat’
         msg_Dbg( p_dec, "available %sware decoder output format %d (%s)",
         ^
In file included from ../include/vlc_common.h:908:0,
                 from codec/avcodec/video.c:32:
codec/avcodec/video.c:1325:51: error: dereferencing pointer to incomplete type
                  hwaccel ? "hard" : "soft", pi_fmt[i], dsc->name );
                                                   ^
../include/vlc_messages.h:78:63: note: in definition of macro ‘msg_Dbg’
     vlc_Log( VLC_OBJECT(p_this), VLC_MSG_DBG,  MODULE_STRING, __VA_ARGS__ )
                                                               ^
codec/avcodec/video.c:1344:5: error: invalid use of undefined type ‘enum PixelFormat’
     for( size_t i = 0; pi_fmt[i] != PIX_FMT_NONE; i++ )
     ^
codec/avcodec/video.c:1344:30: error: dereferencing pointer to incomplete type
     for( size_t i = 0; pi_fmt[i] != PIX_FMT_NONE; i++ )
                              ^
codec/avcodec/video.c:1346:9: error: invalid use of undefined type ‘enum PixelFormat’
         if( p_va->pix_fmt != pi_fmt[i] )
         ^
codec/avcodec/video.c:1346:36: error: dereferencing pointer to incomplete type
         if( p_va->pix_fmt != pi_fmt[i] )
                                    ^
codec/avcodec/video.c:1370:9: error: invalid use of undefined type ‘enum PixelFormat’
         return pi_fmt[i];
         ^
codec/avcodec/video.c:1370:22: error: dereferencing pointer to incomplete type
         return pi_fmt[i];
                      ^
codec/avcodec/video.c:1370:9: warning: ‘return’ with a value, in function returning void
         return pi_fmt[i];
         ^
codec/avcodec/video.c:1378:51: warning: passing argument 2 of ‘avcodec_default_get_format’ from incompatible pointer type
     return avcodec_default_get_format( p_context, pi_fmt );
                                                   ^
In file included from codec/avcodec/video.c:38:0:
/gnu/store/p202gwk3i870jwdwa72j08x9q8n1n2qs-ffmpeg-3.0/include/libavcodec/avcodec.h:5010:20: note: expected ‘const enum AVPixelFormat *’ but argument is of type ‘const enum Pix
elFormat *’
 enum AVPixelFormat avcodec_default_get_format(struct AVCodecContext *s, const enum AVPixelFormat * fmt);
                    ^
codec/avcodec/video.c:1378:5: warning: ‘return’ with a value, in function returning void
     return avcodec_default_get_format( p_context, pi_fmt );
     ^
codec/avcodec/video.c: At top level:
codec/avcodec/video.c:111:25: warning: ‘ffmpeg_GetFormat’ used but never defined
 static enum PixelFormat ffmpeg_GetFormat( AVCodecContext *,
                         ^
codec/avcodec/video.c:1305:25: warning: ‘ffmpeg_GetFormat’ defined but not used [-Wunused-function]
 static enum PixelFormat ffmpeg_GetFormat( AVCodecContext *p_context,
                         ^
cc1: some warnings being treated as errors
Makefile:7735: recipe for target 'codec/avcodec/libavcodec_plugin_la-video.lo' failed
make[4]: *** [codec/avcodec/libavcodec_plugin_la-video.lo] Error 1
make[4]: *** Waiting for unfinished jobs....
make[4]: Leaving directory '/tmp/guix-build-vlc-2.2.1.drv-0/vlc-2.2.1/modules'
Makefile:9652: recipe for target 'all-recursive' failed
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory '/tmp/guix-build-vlc-2.2.1.drv-0/vlc-2.2.1/modules'
Makefile:4526: recipe for target 'all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory '/tmp/guix-build-vlc-2.2.1.drv-0/vlc-2.2.1/modules'
Makefile:2262: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/tmp/guix-build-vlc-2.2.1.drv-0/vlc-2.2.1'
Makefile:2147: recipe for target 'all' failed
make: *** [all] Error 2
phase `build' failed after 44.6 seconds
builder for `/gnu/store/bpa68ylz20k6mczni69w2dmfcbkf12a7-vlc-2.2.1.drv' failed wi
th exit code 1
@ build-failed /gnu/store/bpa68ylz20k6mczni69w2dmfcbkf12a7-vlc-2.2.1.drv - 1 buil
der for `/gnu/store/bpa68ylz20k6mczni69w2dmfcbkf12a7-vlc-2.2.1.drv' failed with e
xit code 1
guix build: error: build failed: build of `/gnu/store/bpa68ylz20k6mczni69w2dmfcbk
f12a7-vlc-2.2.1.drv' failed