golang编程之逐行处理文件

    上篇博文学习了go语言的对FILE的基本操作，我突然想到，文件一个很常用的场景是逐行处理，比如我们的linux下的神器awk，比如我之前写的KMean++算法处理NBA后卫的数据。对于C语言而言，fgets就解决了这个问题，看下C语言中fgets的接口：    

char *fgets(char *s, int size, FILE *stream);

    当然了首先要fopen，获得文件描述符，然后可以fgets按行获取。
    我给出个C程序，完成基本的cat功能，支持-n选项，带了-n则打印出行号：

manu@manu-hacks:~/code/c/self/readline$ cat mycat.c
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<errno.h>



int num_flag = 0;

int cat(FILE* file)
{
    char buf[1024] = {0};
    int line_no = 1;
    while(fgets(buf,1024,file) != NULL)
    {
        if(num_flag != 0)
        {
            fprintf(stdout,"%5d %s",line_no,buf);
        }
        else
        {
            fprintf(stdout,"%s",buf);
        }
        line_no++;
    }

}

int main(int argc,char* argv[])
{
    int i = 0 ;
    int j = 0 ;
    int file_exist = 0;
    FILE* file = NULL;

    for(i = 1; i < argc;i++)
    {
        if(strcmp(argv[i],"-n") == 0)
        {
            num_flag = 1;
            break;
        }
    }

    for(j = 1; j<argc ;j++)
    {
        if(j==i)
            continue;

        file_exist = 1;

        file = fopen(argv[j],"rb");
        if(file == NULL)
        {
            fprintf(stderr,"%s:err reading from %s:%s\n",
                    argv[0],argv[j],strerror(errno));
            continue;
        }

        cat(file);
    }

    if(file_exist == 0)
    {
        cat(stdin);
    }
}

    golang怎么办?
    golang 提供了package bufio。bufio.NewReader()创建一个默认大小的readbuf，当然，也可以bufio.NewReaderSize

func NewReader(rd io.Reader) *Reader
    NewReader returns a new Reader whose buffer has the default size(4096).


func NewReaderSize(rd io.Reader, size int) *Reader
    NewReaderSize returns a new Reader whose buffer has at least the
    specified size. If the argument io.Reader is already a Reader with large
    enough size, it returns the underlying Reader.

    bufio提供

func (b *Reader) ReadByte() (c byte, err error)
    ReadByte reads and returns a single byte. If no byte is available,
    returns an error.

func (b *Reader) ReadBytes(delim byte) (line []byte, err error)
    ReadBytes reads until the first occurrence of delim in the input,
    returning a slice containing the data up to and including the delimiter.
    If ReadBytes encounters an error before finding a delimiter, it returns
    the data read before the error and the error itself (often io.EOF).
    ReadBytes returns err != nil if and only if the returned data does not
    end in delim. For simple uses, a Scanner may be more convenient.

func (b *Reader) ReadString(delim byte) (line string, err error)
    ReadString reads until the first occurrence of delim in the input,
    returning a string containing the data up to and including the
    delimiter. If ReadString encounters an error before finding a delimiter,
    it returns the data read before the error and the error itself (often
    io.EOF). ReadString returns err != nil if and only if the returned data
    does not end in delim. For simple uses, a Scanner may be more
    convenient.

    ReadByte这个接口，和C语言中fgetc很接近，每次读取一个字节。ReadBytes和ReadString都可以实现逐行读取，只要delim设置为'\n'.
    看一下go语言实现的简易mycat：    

manu@manu-hacks:~/code/go/self$ cat mycat.go
package main
import "fmt"
import "os"
import "io"
import "flag"
import "bufio"

var num_flag = flag.Bool("n",false,"num each line")

func usage(){
    fmt.Printf("%s %s\n",os.Args[0],"filename")
}



func cat(r *bufio.Reader){
    i := 1
    for {
        //buf,err := r.ReadBytes('\n')
        buf,err := r.ReadString('\n')
        if err == io.EOF{
            break
        }

        if *num_flag{
            fmt.Fprintf(os.Stdout,"%5d %s",
                        i,buf)
            i++
        }else{
            fmt.Fprintf(os.Stdout,"%s",buf)
        }

    }
    return
}


func main(){

    flag.Parse()
    if(flag.NArg() == 0){
        cat(bufio.NewReader(os.Stdin))
    }

    for i:=0;i<flag.NArg();i++{
        f,err := os.OpenFile(flag.Arg(i),os.O_RDONLY,0660)
        if err != nil{
            fmt.Fprintf(os.Stderr,"%s err read from %s : %s\n",
            os.Args[0],flag.Arg(0),err)
            continue
        }

        cat(bufio.NewReader(f))
        f.Close()
    }
}

    单纯考虑逐行读取，line by line， bufio的文档也说

For simple uses, a Scanner may be more convenient.

    先看文档：

func NewScanner(r io.Reader) *Scanner
    NewScanner returns a new Scanner to read from r. The split function
    defaults to ScanLines.

func (s *Scanner) Text() string
    Text returns the most recent token generated by a call to Scan as a
    newly allocated string holding its bytes.

func (s *Scanner) Err() error
    Err returns the first non-EOF error that was encountered by the Scanner.

func (s *Scanner) Scan() bool
    Scan advances the Scanner to the next token, which will then be
    available through the Bytes or Text method. It returns false when the
    scan stops, either by reaching the end of the input or an error. After
    Scan returns false, the Err method will return any error that occurred
    during scanning, except that if it was io.EOF, Err will return nil.

    怎么用Scanne呢?   

func cat(scanner *bufio.Scanner) error{

    for scanner.Scan(){
        fmt.Println(scanner.Text())
      //fmt.Fprintf(os.Stdout,"%s\n",scanner.Text())
    }

    return  scanner.Err()
}

    注意，为啥执行Scan，Text()函数就能返回下一行呢？因为默认的分割函数就是ScanLines.如你有特殊的需求来分割，func (s *Scanner) Split(split SplitFunc)

这个函数可以制定SplitFunc。你可以定制自己的分割函数。

    需要注意的是，Scan会将分割符号\n去除，如果Fprintf输出的话，不添加\n打印，会出现没有换行的现象，如下所示

fmt.Fprintf(os.Stdout,"%s",scanner.Text())

manu@manu-hacks:~/code/go/self$ go run mycat_v2.go  test.txt
this is test file created by goif not existed ,please create this fileif existed, Please write appendhello world,hello gothis is test file created by goif not existed ,please create this fileif existed, Please write appendhello world,hello gomanu@manu-hacks:~/code/go/self$ cat test.txt
this is test file created by go
if not existed ,please create this file
if existed, Please write append
hello world,hello go
this is test file created by go
if not existed ,please create this file
if existed, Please write append
hello world,hello go

    调用部分的代码如下：

        f,err := os.OpenFile(flag.Arg(i),os.O_RDONLY,0660)
                 ...
        error := cat(bufio.NewScanner(f))
        if err != nil{
            fmt.Fprintf(os.Stderr,"%s err read from %s : %s\n",
            os.Args[0],flag.Arg(i),error)
        }

    推荐使用Scanner，使用比较简单。
参考文献：
1 godoc bufio
2