Monday, December 6, 2010

Design Assumptions of Google File System:

  • •
    components that often fail. It must constantly monitor
    itself and detect, tolerate, and recover promptly from
    component failures on a routine basis.
    •
    expect a few million files, each typically 100 MB or
    larger in size. Multi-GB files are the common case
    and should be managed efficiently. Small files must be
    supported, but we need not optimize for them.
    •
    large streaming reads and small random reads. In
    large streaming reads, individual operations typically
    read hundreds of KBs, more commonly 1 MB or more.
    Successive operations from the same client often read
    through a contiguous region of a file. A small random
    read typically reads a few KBs at some arbitrary
    offset. Performance-conscious applications often batch
    and sort their small reads to advance steadily through
    the file rather than go backan d forth.
    •
    that append data to files. Typical operation sizes are
    similar to those for reads. Once written, files are seldom
    modified again. Small writes at arbitrary positions
    in a file are supported but do not have to be
    efficient.
    •
    for multiple clients that concurrently append
    to the same file. Our files are often used as producerconsumer
    queues or for many-way merging. Hundreds
    of producers, running one per machine, will concurrently
    append to a file. Atomicity with minimal synchronization
    overhead is essential. The file may be
    read later, or a consumer may be reading through the
    file simultaneously.
    •
    latency. Most of our target applications place a premium
    on processing data in bulka t a high rate, while
    few have stringent response time requirements for an
    individual read or write.
    High sustained bandwidth is more important than low
    The system must efficiently implement well-defined semantics
    The workloads also have many large, sequential writes
    The workloads primarily consist of two kinds of reads:
    The system stores a modest number of large files. We
    The system is built from many inexpensive commodity

No comments:

Post a Comment