Our Experiences on DirectFB in Embedded Application Development

本文根据 IGEL 公司和 Renesas 公司的工程师做的 presentation 整理而成，介绍了他们使用 DirectFB 开发嵌入式应用程序的经验，内容包括 DirectFB 的体系结构、硬件加速带来的性能提升、GFX graphics 驱动开发方法、DirectFB 缺少的 features 等内容。本文图文并茂，对使用者较深入的理解 DirectFB 有很大帮助。
体系结构体系结构如下图所示：

• works on a frame buffer device(/dev/fb) and provides the mechanism to use the hardware acceleration effectively.
• consists of the followings:
  1. Core API Module;
  2. Generic GFX Driver;
  3. GFX Drivers for Specific Hardware.
• To bring out the best performance on a specific graphics hardware, GFX Drivers for the hardware should be written.
• Generic GFX Driver checks whether the hardware acceleration by a GFX driver is available:
  1. If yes, it handovers to the GFX driver;
  2. If not, it uses software rendering engine.

Why do we need GFX driver

• Embedded CPU and bus are slow compare to Desktop’s CPU;
  1. 200-400MHz CPU;
  2. 120MHz 32bit Bus.
• Therefore, handover the rendering tasks to specialized hardware is crucial.

Effects of Hardware Acceleration

• The hardware acceleration shows remarkable results.
• The performance depends on hardware acceleration engine rather than CPU.

How to write GFX Driver

Callback routines needs to be written

1. GFX Driver Functions;
2. GFX Device Functions.

Good starting point is gfxdrivers/i810/*.[ch]

GFX Driver Function：

From core/gfxcard.h:

typedef struct {
     int       (*Probe)          (GraphicsDevice      *device);
     void      (*GetDriverInfo)  (GraphicsDevice      *device,
                                  GraphicsDriverInfo  *driver_info);

     DFBResult (*InitDriver)     (GraphicsDevice      *device,
                                  GraphicsDeviceFuncs *funcs,
                                  void                *driver_data,
                                  void                *device_data);

     DFBResult (*InitDevice)     (GraphicsDevice      *device,
                                  GraphicsDeviceInfo  *device_info,
                                  void                *driver_data,
                                  void                *device_data);

     void      (*CloseDevice)    (GraphicsDevice      *device,
                                  void                *driver_data,
                                  void                *device_data);
     void      (*CloseDriver)    (GraphicsDevice      *device,
                                  void                *driver_data);
} GraphicsDriverFuncs;

GFX Device Functions：

From core/gfxcard.h:
typedef struct _GraphicsDeviceFuncs {
     /*
      * function that is called after variable screeninfo is changed
      * (used for buggy fbdev drivers, that reinitialize something when
      * calling FBIO_PUT_VSCREENINFO)
      */
     void (*AfterSetVar)( void *driver_data, void *device_data );

     /*
      * Called after driver->InitDevice() and during dfb_gfxcard_unlock( true ).
      * The driver should do the one time initialization of the engine,
      * e.g. writing some registers that are supposed to have a fixed value.
      *
      * This happens after mode switching or after returning from
      * OpenGL state (e.g. DRI driver).
      */
     void (*EngineReset)( void *driver_data, void *device_data );

     /*
      * Makes sure that graphics hardware has finished all operations.
      *
      * This method is called before the CPU accesses a surface' buffer
      * that had been written to by the hardware after this method has been
      * called the last time.
      *
      * It's also called before entering the OpenGL state (e.g. DRI driver).
      */
     void (*EngineSync)( void *driver_data, void *device_data );

     /*
      * after the video memory has been written to by the CPU (e.g. modification
      * of a texture) make sure the accelerator won't use cached texture data
      */
     void (*FlushTextureCache)( void *driver_data, void *device_data );

     /*
      * Check if the function 'accel' can be accelerated with the 'state'.
      * If that's true, the function sets the 'accel' bit in 'state->accel'.
      * Otherwise the function just returns, no need to clear the bit.
      */
     void (*CheckState)( void *driver_data, void *device_data,
                     CardState *state, DFBAccelerationMask accel );

     /*
      * Program card for execution of the function 'accel' with the 'state'.
      * 'state->modified' contains information about changed entries.
      * This function has to set at least 'accel' in 'state->set'.
      * The driver should remember 'state->modified' and clear it.
      * The driver may modify 'funcs' depending on 'state' settings.
      */
     void (*SetState)  ( void *driver_data, void *device_data,
                     struct _GraphicsDeviceFuncs *funcs,
                     CardState *state, DFBAccelerationMask accel );
     /*
      * drawing functions
      */
     bool (*FillRectangle) ( void *driver_data, void *device_data,
                             DFBRectangle *rect );

     bool (*DrawRectangle) ( void *driver_data, void *device_data,
                             DFBRectangle *rect );

     bool (*DrawLine)      ( void *driver_data, void *device_data,
                             DFBRegion *line );

     bool (*FillTriangle)  ( void *driver_data, void *device_data,
                             DFBTriangle *tri );

     /*
      * blitting functions
      */
     bool (*Blit)          ( void *driver_data, void *device_data,
                             DFBRectangle *rect, int dx, int dy );

     bool (*StretchBlit)   ( void *driver_data, void *device_data,
                             DFBRectangle *srect, DFBRectangle *drect );

     /*
      * emit any buffered commands, i.e. trigger processing
      */
     void (*EmitCommands)  ( void *driver_data, void *device_data );
} GraphicsDeviceFuncs;

Porting on SH7751 + SM501

需要开发为 SM501 开发 GFX Driver。

基本思路如下图所示：

GFX Driver for SM501 just set registers to issue rendering commands:

1. Issuing command is done on the fly;
2. Callback functions immediately set registers to render;
3. Rendering comes on screen instantly.

Porting on SH7770

SH7770 的驱动比上面的 SM501 更复杂一些。

GFX Driver for SH7770 creates list of rendering commands, so called Display List.The list is double buffered.

The driver fills the list until they’re full, and then pass them to the 2D engine.

1. While the driver is filling one list, the 2D engine reads commands from another list;
2. Once the 2D engine is done with the list, it sends an interrupt,and get the next list;
3. Rendering doesn’t come on screen instantly.

Sync mechanism is required to sync with software rendering done by generic GFX driver

Missing Pieces in

• Access multiple layered frame buffer from a process.Recent graphics hardware has multiple frame buffers;
• Using ‘scroll’ function on hardware. New feature not covered by API;
• Synchronous rendering and display with VSYNC (QoS, delay handling).Real-time motion graphics (e.g. game), car navigation, etc;
• Synchronize 2D Engine and 3D Engine.
  1. Render with 2D and 3D Engine in a single layer;
  2. Synchronous display even 2D and 3D are on different layers.

Access Multiple Layered Frame Buffer from a Process

Recent graphics hardware has multiple layered frame buffer.

To coordinate layers efficiently, an application process wants to issue rendering commands and switch on / off display of each layer.

Using "Scroll" Function on Hardware

Use Case: car navigation system, web browser.

The 'Scroll' function reduces re-rendering cost.

Synchronous Rendering and Display with VSYNC (QoS)

In real-time motion graphics applications, the screen must be updated in sync with the VSYNC signal.

Under the standard (fairly) task scheduling in Linux, rendering might miss display timing (VSYNC) because of signal interrupts or other heavy tasks.

Real-time motion graphics applications could be optimized for screen rendering, especially for delay handling.

Application needs VSYNC signal timing to notice the delay.

Delay Handling by Application:

1. Skipping the next frame rendering.When an application noticed that display has been delayed, it could skip the next frame and start rendering the frame after the next.
2. Updating screen even if the rendering is not finished.Additionally, application could give priority to rendering operations in case of incomplete frame displaying.

Synchronize 2D Engine and 3D Engine

Many 3D graphics applications combine 3D graphics and 2D graphics.

These 2D graphics must be synchronized with 3D graphics.

Some 3D acceleration hardware (nVIDIA, ATI, for example) are separated from 2D hardware.

Synchronization mechanism between 2D and 3D graphics (hardware) is needed.

Situation 1: Rendering 2D graphics and 3D graphics into a single layer simultaneously

1. 2D engine and 3D engine tries to draw into a frame without any synchronization. They should be serialized.
2. Issued rendering commands might be performed asynchronously.

Situation 2: Displaying a 2D graphics layer and a 3D graphics layer synchronously.

1. 2D/3D engines can draw into each independent layer asynchronously if the multiple layered frame buffer is available.
2. Synchronous display function is still needed.