How to Concatenate (cat and compose) Buckets with the gsutil Utility

The cat command in the gsutil utility directs an object's text to standard output. For example, the following command prints the text contained in gs://mybucket/a.txt:

gsutil cat gs://mybucket/a.txt

Despite its name, you can’t concatenate objects with cat, but you can concatenate objects with compose. That is, the following command concatenates the content of a.txt and b.txt in gs://mybucket and stores the combined result to c.txt:

gsutil compose gs://mybucket/a.txt gs://mybucket/b.txt gs://mybucket/c.txt

When you use compose, keep three points in mind:

• A project can perform at most 200 compose operations per second.
• A compose operation can combine a maximum of 32 entities.
• A given object can be appended to at most 1,023 times.

compose is particularly helpful if you have to upload very large files to Cloud Storage. Rather than upload an entire file from one computer, you can upload portions of the file from separate computers and use compose to combine the portions.