Class ByteStreams

java.lang.Object
com.google.common.io.ByteStreams

public final class ByteStreams extends Object
Provides utility methods for working with byte arrays and I/O streams.
Since:
1.0
  • Field Details

    • BUFFER_SIZE

      private static final int BUFFER_SIZE
      See Also:
    • ZERO_COPY_CHUNK_SIZE

      private static final int ZERO_COPY_CHUNK_SIZE
      There are three methods to implement FileChannel.transferTo(long, long, WritableByteChannel):
      1. Use sendfile(2) or equivalent. Requires that both the input channel and the output channel have their own file descriptors. Generally this only happens when both channels are files or sockets. This performs zero copies - the bytes never enter userspace.
      2. Use mmap(2) or equivalent. Requires that either the input channel or the output channel have file descriptors. Bytes are copied from the file into a kernel buffer, then directly into the other buffer (userspace). Note that if the file is very large, a naive implementation will effectively put the whole file in memory. On many systems with paging and virtual memory, this is not a problem - because it is mapped read-only, the kernel can always page it to disk "for free". However, on systems where killing processes happens all the time in normal conditions (i.e., android) the OS must make a tradeoff between paging memory and killing other processes - so allocating a gigantic buffer and then sequentially accessing it could result in other processes dying. This is solvable via madvise(2), but that obviously doesn't exist in java.
      3. Ordinary copy. Kernel copies bytes into a kernel buffer, from a kernel buffer into a userspace buffer (byte[] or ByteBuffer), then copies them from that buffer into the destination channel.
      This value is intended to be large enough to make the overhead of system calls negligible, without being so large that it causes problems for systems with atypical memory management if approaches 2 or 3 are used.
      See Also:
    • MAX_ARRAY_LEN

      private static final int MAX_ARRAY_LEN
      Max array length on JVM.
      See Also:
    • TO_BYTE_ARRAY_DEQUE_SIZE

      private static final int TO_BYTE_ARRAY_DEQUE_SIZE
      Large enough to never need to expand, given the geometric progression of buffer sizes.
      See Also:
    • NULL_OUTPUT_STREAM

      private static final OutputStream NULL_OUTPUT_STREAM
  • Constructor Details

    • ByteStreams

      private ByteStreams()
  • Method Details

    • createBuffer

      static byte[] createBuffer()
      Creates a new byte array for buffering reads or writes.
    • copy

      public static long copy(InputStream from, OutputStream to) throws IOException
      Copies all bytes from the input stream to the output stream. Does not close or flush either stream.

      Java 9 users and later: this method should be treated as deprecated; use the equivalent InputStream.transferTo(java.io.OutputStream) method instead.

      Parameters:
      from - the input stream to read from
      to - the output stream to write to
      Returns:
      the number of bytes copied
      Throws:
      IOException - if an I/O error occurs
    • copy

      public static long copy(ReadableByteChannel from, WritableByteChannel to) throws IOException
      Copies all bytes from the readable channel to the writable channel. Does not close or flush either channel.
      Parameters:
      from - the readable channel to read from
      to - the writable channel to write to
      Returns:
      the number of bytes copied
      Throws:
      IOException - if an I/O error occurs
    • toByteArrayInternal

      private static byte[] toByteArrayInternal(InputStream in, Queue<byte[]> bufs, int totalLen) throws IOException
      Returns a byte array containing the bytes from the buffers already in bufs (which have a total combined length of totalLen bytes) followed by all bytes remaining in the given input stream.
      Throws:
      IOException
    • combineBuffers

      private static byte[] combineBuffers(Queue<byte[]> bufs, int totalLen)
    • toByteArray

      public static byte[] toByteArray(InputStream in) throws IOException
      Reads all bytes from an input stream into a byte array. Does not close the stream.

      Java 9+ users: use in#readAllBytes() instead.

      Parameters:
      in - the input stream to read from
      Returns:
      a byte array containing all the bytes from the stream
      Throws:
      IOException - if an I/O error occurs
    • toByteArray

      static byte[] toByteArray(InputStream in, long expectedSize) throws IOException
      Reads all bytes from an input stream into a byte array. The given expected size is used to create an initial byte array, but if the actual number of bytes read from the stream differs, the correct result will be returned anyway.
      Throws:
      IOException
    • exhaust

      public static long exhaust(InputStream in) throws IOException
      Reads and discards data from the given InputStream until the end of the stream is reached. Returns the total number of bytes read. Does not close the stream.
      Throws:
      IOException
      Since:
      20.0
    • newDataInput

      public static ByteArrayDataInput newDataInput(byte[] bytes)
      Returns a new ByteArrayDataInput instance to read from the bytes array from the beginning.
    • newDataInput

      public static ByteArrayDataInput newDataInput(byte[] bytes, int start)
      Returns a new ByteArrayDataInput instance to read from the bytes array, starting at the given position.
      Throws:
      IndexOutOfBoundsException - if start is negative or greater than the length of the array
    • newDataInput

      public static ByteArrayDataInput newDataInput(ByteArrayInputStream byteArrayInputStream)
      Returns a new ByteArrayDataInput instance to read from the given ByteArrayInputStream. The given input stream is not reset before being read from by the returned ByteArrayDataInput.
      Since:
      17.0
    • newDataOutput

      public static ByteArrayDataOutput newDataOutput()
      Returns a new ByteArrayDataOutput instance with a default size.
    • newDataOutput

      public static ByteArrayDataOutput newDataOutput(int size)
      Returns a new ByteArrayDataOutput instance sized to hold size bytes before resizing.
      Throws:
      IllegalArgumentException - if size is negative
    • newDataOutput

      public static ByteArrayDataOutput newDataOutput(ByteArrayOutputStream byteArrayOutputStream)
      Returns a new ByteArrayDataOutput instance which writes to the given ByteArrayOutputStream. The given output stream is not reset before being written to by the returned ByteArrayDataOutput and new data will be appended to any existing content.

      Note that if the given output stream was not empty or is modified after the ByteArrayDataOutput is created, the contract for ByteArrayDataOutput.toByteArray() will not be honored (the bytes returned in the byte array may not be exactly what was written via calls to ByteArrayDataOutput).

      Since:
      17.0
    • nullOutputStream

      public static OutputStream nullOutputStream()
      Returns an OutputStream that simply discards written bytes.
      Since:
      14.0 (since 1.0 as com.google.common.io.NullOutputStream)
    • limit

      public static InputStream limit(InputStream in, long limit)
      Wraps a InputStream, limiting the number of bytes which can be read.
      Parameters:
      in - the input stream to be wrapped
      limit - the maximum number of bytes to be read
      Returns:
      a length-limited InputStream
      Since:
      14.0 (since 1.0 as com.google.common.io.LimitInputStream)
    • readFully

      public static void readFully(InputStream in, byte[] b) throws IOException
      Attempts to read enough bytes from the stream to fill the given byte array, with the same behavior as DataInput.readFully(byte[]). Does not close the stream.
      Parameters:
      in - the input stream to read from.
      b - the buffer into which the data is read.
      Throws:
      EOFException - if this stream reaches the end before reading all the bytes.
      IOException - if an I/O error occurs.
    • readFully

      public static void readFully(InputStream in, byte[] b, int off, int len) throws IOException
      Attempts to read len bytes from the stream into the given array starting at off, with the same behavior as DataInput.readFully(byte[], int, int). Does not close the stream.
      Parameters:
      in - the input stream to read from.
      b - the buffer into which the data is read.
      off - an int specifying the offset into the data.
      len - an int specifying the number of bytes to read.
      Throws:
      EOFException - if this stream reaches the end before reading all the bytes.
      IOException - if an I/O error occurs.
    • skipFully

      public static void skipFully(InputStream in, long n) throws IOException
      Discards n bytes of data from the input stream. This method will block until the full amount has been skipped. Does not close the stream.
      Parameters:
      in - the input stream to read from
      n - the number of bytes to skip
      Throws:
      EOFException - if this stream reaches the end before skipping all the bytes
      IOException - if an I/O error occurs, or the stream does not support skipping
    • skipUpTo

      static long skipUpTo(InputStream in, long n) throws IOException
      Discards up to n bytes of data from the input stream. This method will block until either the full amount has been skipped or until the end of the stream is reached, whichever happens first. Returns the total number of bytes skipped.
      Throws:
      IOException
    • skipSafely

      private static long skipSafely(InputStream in, long n) throws IOException
      Attempts to skip up to n bytes from the given input stream, but not more than in.available() bytes. This prevents FileInputStream from skipping more bytes than actually remain in the file, something that it specifies it can do in its Javadoc despite the fact that it is violating the contract of InputStream.skip().
      Throws:
      IOException
    • readBytes

      public static <T> T readBytes(InputStream input, ByteProcessor<T> processor) throws IOException
      Process the bytes of the given input stream using the given processor.
      Parameters:
      input - the input stream to process
      processor - the object to which to pass the bytes of the stream
      Returns:
      the result of the byte processor
      Throws:
      IOException - if an I/O error occurs
      Since:
      14.0
    • read

      public static int read(InputStream in, byte[] b, int off, int len) throws IOException
      Reads some bytes from an input stream and stores them into the buffer array b. This method blocks until len bytes of input data have been read into the array, or end of file is detected. The number of bytes read is returned, possibly zero. Does not close the stream.

      A caller can detect EOF if the number of bytes read is less than len. All subsequent calls on the same stream will return zero.

      If b is null, a NullPointerException is thrown. If off is negative, or len is negative, or off+len is greater than the length of the array b, then an IndexOutOfBoundsException is thrown. If len is zero, then no bytes are read. Otherwise, the first byte read is stored into element b[off], the next one into b[off+1], and so on. The number of bytes read is, at most, equal to len.

      Parameters:
      in - the input stream to read from
      b - the buffer into which the data is read
      off - an int specifying the offset into the data
      len - an int specifying the number of bytes to read
      Returns:
      the number of bytes read
      Throws:
      IOException - if an I/O error occurs
      IndexOutOfBoundsException - if off is negative, if len is negative, or if off + len is greater than b.length