yaml解析实例

Introduction

This tutorial is an introduction to using the libyaml library with the C programming language. It assumes a basic knowledge of the YAML format. See  for more information and links to the relevant sites.

More Information

All information used to write this page came from the header , and trial-and-error.

YAML

What is YAML?

YAML stands for ``YAML Ain't a Markup Language'', and is a data-storage format designed for easy readability and machine parsing. It is more verbose than JSON, and less verbose than XML, two other formats designed for similar purposes.

It allows data to be nested arbitrarily deep, allows embedded newlines, and supports sequences, relational trees and all sorts of fancy stuff. I'll be shying away from the advanced features and sticking to the basics.

What Does it Look Like?

I am no YAML expert. I am learning the language as I write this — in fact, before this morning I had never used YAML, though I'd heard of it. Arguably, I am the last person who should be writing this tutorial. But last I looked, there was no decent beginner's tutorial for libyaml, so I'm doing it.

Here is the file I will be parsing:

# config/public.yaml

title   : Finex 2011
img_url : /finex/html/img/
css_url : /finex/html/style/
js_url  : /finex/html/js/

template_dir: html/templ/

default_act : idx    # used for invalid/missing act=

pages:
  - act   : idx
    title : Welcome
    html  : public/welcome.phtml
  - act   : reg
    title : Register
    html  : public/register.phtml
  - act   : log
    title : Log in
    html  : public/login.phtml
  - act   : out
    title : Log out
    html  : public/logout.phtml

One non-obvious (from this file) thing about YAML is that it forbids the tab character. Why? Because YAML depends on indentation for its structure, and tabs tend to mess things up.

This is part of my ongoing project to port Finex away from PHP. (Why C? I'm trying quite a few languages, and C happens to be today's.) As you can see, this is a simple file I'll be using to instruct my templating system on how to read the act=XXX portion of a URL.

libyaml

libyaml is a C library for parsing YAML files, and probably available from your package manager. To use it, include the file yaml.h and add the linker flag -lyaml to gcc.

For detailed information, check out the header file /usr/include/yaml.h.

yaml_parser_t

The primary object used by libyaml is the parser itself. This is an object of type yaml_parser_t. It must be allocated manually (usually on the stack) and is initialized/deinitialized by the functions:

int yaml_parser_initialize(yaml_parser_t *) void yaml_parser_delete(yaml_parser_t *)

All error codes are returned as ints. 1 signifies success, 0 failure. Next, to open a specific file, we use the function:

void yaml_parser_set_input_file(yaml_parser_t *parser, FILE *file)

There are also functions to read input from a string or generic read handler, and to set the encoding of an input file. We won't cover those here, but be aware.

Our code thus far is:

#include  #include  int main(void)
{ FILE *fh = fopen("config/public.yaml", "r");
  yaml_parser_t parser; /* Initialize parser */ if(!yaml_parser_initialize(&parser))
    fputs("Failed to initialize parser!\n", stderr); if(fh == NULL)
    fputs("Failed to open file!\n", stderr); /* Set input file */ yaml_parser_set_input_file(&parser, fh); /* CODE HERE */ /* Cleanup */ yaml_parser_delete(&parser);
  fclose(fh); return 0;
}

This should compile and run without error, though it doesn't do anything yet.

Token-Based and Stream-Based Parsing

There are two ways to parse a YAML document using libyaml: token-based and event-based. The simplest way, conceptually, is token-based. By using the functions

int yaml_parser_scan(yaml_parser_t *parser, yaml_token_t *token) void yaml_token_delete(yaml_token_t *token)

we can get each token from the YAML document in turn. The full yaml_token_t structure can be found in yaml.h, but for our purposes we will only need the two fields .type and .data.scalar.value, which tell us the token type, and its data (if the type is YAML_SCALAR_TOKEN).

Though I do not cover every token type – again, see yaml.h for more – the following example code should be illustrative:

#include  #include  int main(void)
{ FILE *fh = fopen("config/public.yaml", "r");
  yaml_parser_t parser;
  yaml_token_t  token; /* new variable */ /* Initialize parser */ if(!yaml_parser_initialize(&parser))
    fputs("Failed to initialize parser!\n", stderr); if(fh == NULL)
    fputs("Failed to open file!\n", stderr); /* Set input file */ yaml_parser_set_input_file(&parser, fh); /* BEGIN new code */ do {
    yaml_parser_scan(&parser, &token); switch(token.type)
    { /* Stream start/end */ case YAML_STREAM_START_TOKEN: puts("STREAM START"); break; case YAML_STREAM_END_TOKEN:   puts("STREAM END");   break; /* Token types (read before actual token) */ case YAML_KEY_TOKEN:   printf("(Key token)   "); break; case YAML_VALUE_TOKEN: printf("(Value token) "); break; /* Block delimeters */ case YAML_BLOCK_SEQUENCE_START_TOKEN: puts("Start Block (Sequence)"); break; case YAML_BLOCK_ENTRY_TOKEN:          puts("Start Block (Entry)");    break; case YAML_BLOCK_END_TOKEN:            puts("End block");              break; /* Data */ case YAML_BLOCK_MAPPING_START_TOKEN:  puts("[Block mapping]");            break; case YAML_SCALAR_TOKEN:  printf("scalar %s \n", token.data.scalar.value); break; /* Others */ default:
      printf("Got token of type %d\n", token.type);
    } if(token.type != YAML_STREAM_END_TOKEN)
      yaml_token_delete(&token);
  } while(token.type != YAML_STREAM_END_TOKEN);
  yaml_token_delete(&token); /* END new code */ /* Cleanup */ yaml_parser_delete(&parser);
  fclose(fh); return 0;
}

This simple loop reads every token from the document and prints it out. The output forpublic.yaml, indented to show block structure, is:

STREAM START  
  [Block mapping] 
  (Key token)   scalar title 
  (Value token) scalar Finex 2011 
  (Key token)   scalar img_url 
  (Value token) scalar /finex/html/img/ 
  (Key token)   scalar css_url 
  (Value token) scalar /finex/html/style/ 
  (Key token)   scalar js_url 
  (Value token) scalar /finex/html/js/ 
  (Key token)   scalar template_dir 
  (Value token) scalar html/templ/ 
  (Key token)   scalar pages 
  (Value token) Start Block (Sequence) Start Block (Entry) [Block mapping] 
      (Key token)   scalar act 
      (Value token) scalar idx 
      (Key token)   scalar title 
      (Value token) scalar Welcome 
      (Key token)   scalar html 
      (Value token) scalar public/welcome.phtml End block Start Block (Entry) [Block mapping] 
      (Key token)   scalar act 
      (Value token) scalar reg 
      (Key token)   scalar title 
      (Value token) scalar Register 
      (Key token)   scalar html 
      (Value token) scalar public/register.phtml End block Start Block (Entry) [Block mapping] 
      (Key token)   scalar act 
      (Value token) scalar log 
      (Key token)   scalar title 
      (Value token) scalar Log in 
      (Key token)   scalar html 
      (Value token) scalar public/login.phtml End block Start Block (Entry) [Block mapping] 
      (Key token)   scalar act 
      (Value token) scalar out 
      (Key token)   scalar title 
      (Value token) scalar Log out 
      (Key token)   scalar html 
      (Value token) scalar public/logout.phtml End block End block End block STREAM END  

It is clear that for simple documents, token-based parsing makes sense. However, a more natural paradigm is event-based parsing. This works by the similar functions

int yaml_parser_parse(yaml_parser_t *parser, yaml_event_t *event) void yaml_event_delete(yaml_event_t *event)

The code using these functions, and its output, is as follows:

#include  #include  int main(void)
{ FILE *fh = fopen("config/public.yaml", "r");
  yaml_parser_t parser;
  yaml_event_t  event; /* New variable */ /* Initialize parser */ if(!yaml_parser_initialize(&parser))
    fputs("Failed to initialize parser!\n", stderr); if(fh == NULL)
    fputs("Failed to open file!\n", stderr); /* Set input file */ yaml_parser_set_input_file(&parser, fh); /* START new code */ do { if (!yaml_parser_parse(&parser, &event)) {
       printf("Parser error %d\n", parser.error);
       exit(EXIT_FAILURE);
    } switch(event.type)
    { case YAML_NO_EVENT: puts("No event!"); break; /* Stream start/end */ case YAML_STREAM_START_EVENT: puts("STREAM START"); break; case YAML_STREAM_END_EVENT:   puts("STREAM END");   break; /* Block delimeters */ case YAML_DOCUMENT_START_EVENT: puts("Start Document"); break; case YAML_DOCUMENT_END_EVENT:   puts("End Document");   break; case YAML_SEQUENCE_START_EVENT: puts("Start Sequence"); break; case YAML_SEQUENCE_END_EVENT:   puts("End Sequence");   break; case YAML_MAPPING_START_EVENT:  puts("Start Mapping");  break; case YAML_MAPPING_END_EVENT:    puts("End Mapping");    break; /* Data */ case YAML_ALIAS_EVENT:  printf("Got alias (anchor %s)\n", event.data.alias.anchor); break; case YAML_SCALAR_EVENT: printf("Got scalar (value %s)\n", event.data.scalar.value); break;
    } if(event.type != YAML_STREAM_END_EVENT)
      yaml_event_delete(&event);
  } while(event.type != YAML_STREAM_END_EVENT);
  yaml_event_delete(&event); /* END new code */ /* Cleanup */ yaml_parser_delete(&parser);
  fclose(fh); return 0;
}

STREAM START Start Document Start Mapping Got scalar (value title) 
    Got scalar (value Finex 2011) 
    Got scalar (value img_url) 
    Got scalar (value /finex/html/img/) 
    Got scalar (value css_url) 
    Got scalar (value /finex/html/style/) 
    Got scalar (value js_url) 
    Got scalar (value /finex/html/js/) 
    Got scalar (value template_dir) 
    Got scalar (value html/templ/) 
    Got scalar (value pages) Start Sequence Start Mapping Got scalar (value act) 
        Got scalar (value idx) 
        Got scalar (value title) 
        Got scalar (value Welcome) 
        Got scalar (value html) 
        Got scalar (value public/welcome.phtml) End Mapping Start Mapping Got scalar (value act) 
        Got scalar (value reg) 
        Got scalar (value title) 
        Got scalar (value Register) 
        Got scalar (value html) 
        Got scalar (value public/register.phtml) End Mapping Start Mapping Got scalar (value act) 
        Got scalar (value log) 
        Got scalar (value title) 
        Got scalar (value Log in) 
        Got scalar (value html) 
        Got scalar (value public/login.phtml) End Mapping Start Mapping Got scalar (value act) 
        Got scalar (value out) 
        Got scalar (value title) 
        Got scalar (value Log out) 
        Got scalar (value html) 
        Got scalar (value public/logout.phtml) End Mapping End Sequence End Mapping End Document STREAM END 

There are two major things to notice about this. First, the output is better-structured; the generic ``End Block'' tokens are now specific events. Secondly, the code is simpler. Not only that, but my switch has every event type. As you may recall from the token-based code, the switchstatement was incomplete, and should have contained many  other token types, most of which I don't understand.

Also, an event-based approach is more amenable to object-oriented programming, and so is likely what you'll see when using libyaml with other languages.

Document-Based Parsing

I lied. There is actually a third way to parse YAML, based on the functions

int yaml_parser_load(yaml_parser_t *parser, yaml_document_t *document) void yaml_document_delete(yaml_document_t *document)

These allow you to load individual documents into structures, and manipulate them using a variety of yaml_document_* functions. This is useful because YAML documents may be spread across multiple files, or individual files may contain many YAML documents. However, this is an advanced use case and I haven't looked into it.

.

Looking Forward

This concludes my tutorial. Looking forward, there are a few things to look out for. First, watch out for memory leaks. While YAML does not allocate and return complete objects, any function that populates an existing object will allocate buffers. To avoid leaking, be sure to use the appropriate yaml_*_delete function before re-populating an object.

Also, watch out for versioning. The current YAML version is 1.3, but the libyaml version I'm using only supports up to 1.2. If this is a concern for you, you can use the functions yaml_get_version and yaml_get_version_string to see what your library supports.

Watch out for encoding issues. This is the 21st century, after all. The STREAM_START event and token both have an .encoding field that you can check if you do not know your document encoding in advance.

Finally, you can build YAML trees in code. There are a variety of functions to create your own tokens, events and documents, and the yaml_emitter_t object (and associated functions) will allow you to output files. See yaml.h for details.

January 2011
Andrew Poelstra