IO

My Note

NIO was thought to be faster as it is non-blocking so you don't need to consume so much threads as traditional thread per connection model. Not only that, context switching between threads are not free as well. It sounds all logical to me but I have come across some benchmarks online saying that IO is 25% faster in all cases. Looks like while the software community solved that problem with Java’s NIO libraries, the OS and hardware community solved the original problem of expensive threads with advanced OS threading libraries like NPTL and multi-core machines.

However, is the benchmark on a local laptop can truly reflect what is going to happen in production server environment? If your benchmark reading the same file on disk, you are not testing your java code but your disk only. The reason is the disk cache will kick in to cloud your judgement as you won't see much benefit of NIO if IO cost is extremely low. Just like you don't need NIO solution on top of the memory. Here Stu Thompson performed the benchmark in production server and saw NIO gain 250% in performance comparing to IO.

IO vs NIO (Major Differences)

• Original I/O deals with data in streams (1 byte at a time), whereas NIO deals with data in blocks (1 block at a time). Processing data by the block can be much faster than processing it by the (streamed) byte.
• It is very easy to create filters for streamed data. It is also relatively simply to chain several filters together so that each one does its part in what amounts to a single, sophisticated processing mechanism. Important thing is that bytes are not cached anywhere.
• You cannot move forth and back in the data in a stream. If you need to move forth and back in the data read from a stream, you must cache it in a buffer first. On the other hand, block IO doesn't have this problem.
• Java IO’s various streams are blocking or synchronous. In asynchronous IO, a thread can request that some data be written to a channel, but not wait for it to be fully written.
• Synchronous programs often have to resort to polling, or to the creation of many, many threads, to deal with lots of connections.
• With asynchronous I/O, you can listen for I/O events on an arbitrary number of channels, without polling and without extra threads.

Ref:http://cs.brown.edu/courses/cs161/papers/j-nio-ltr.pdf

I/O Design Patterns

Reactor Pattern

• Reactor initiator - Main
• Demultiplexer - NIO Selector
• Handle - NIO SelectionKey
• Dispatcher - assign the job to a task handler that run by a worker thread from the pool

Proactor Pattern

Ref: http://www.javacodegeeks.com/2012/08/io-demystified.html

Channel and Buffer

Buffer Type

• ByteBuffer, CharBuffer, DoubleBuffer, FloatBuffer, IntBuffer, LongBuffer, ShortBuffer (data type buffer)
• MappedByteBuffer (special)

MappedByteBuffer is a specialization of ByteBuffer used for memory mapped files. None of these classes can be instantiated directly. They are all abstract classes, but each contains static factory methods to create new instances of the appropriate class.

Basic Buffer Usage

• A buffer is essentially a block of memory into which you can write data, which you can then later read again.
• Using a Buffer to read and write data typically follows this little 4-step process:
  • Write data into the Buffer
  • Call buffer.flip() - switch the buffer from writing mode into reading mode using the flip() method call
  • Read data out of the Buffer
  • Call buffer.clear() or buffer.compact() - The clear() method clears the whole buffer. The compact() method only clears the data which you have already read. Any unread data is moved to the beginning of the buffer, and data will now be written into the buffer after the unread data.

RandomAccessFile aFile = new RandomAccessFile("data/nio-data.txt", "rw");
FileChannel inChannel = aFile.getChannel();

//To obtain a Buffer object you must first allocate it. 
ByteBuffer buf = ByteBuffer.allocate(48);


while (bytesRead != -1) {

   System.out.println("Read " + bytesRead);

   //switches a Buffer from writing mode to reading mode
   buf.flip();

   while(buf.hasRemaining()){
       System.out.print((char) buf.get());
   }

   buf.clear();
   bytesRead = inChannel.read(buf);

}
aFile.close();

//get data from buffer into channel
//or you can do: byte aByte = buf.get();
int bytesWritten = inChannel.write(buf);

//Buffer.rewind() sets the position back to 0, so you can reread all the data in the buffer.

//Mark and Reset: You can mark() a given position in a Buffer so you 
//can then later reset() the position back to the marked position
buffer.mark();

//call buffer.get() a couple of times, e.g. during parsing.
buffer.reset();  //set position back to mark.

Write and Read in Detailed

• A Buffer has three properties you need to be familiar with, in order to understand how a Buffer works. These are:
  • capacity (fixed size) - You can only write capacity bytes, longs, chars etc. into the Buffer. Once the Buffer is full, you need to empty it (read the data, or clear it) before you can write more data into it.)
  • position, limit - in write mode, it starts from 0 and advanced to the limit. Once you flip, you will be in read mode. At that time, limit will go to previous write position and position will reset to 0 and advance to the limit during read. So, we will read the data that previously written in sequence.

//Example to copy data from one file to another file

FileInputStream fin = new FileInputStream( "readandshow.txt" );
FileChannel fcin = fin.getChannel();
FileOutputStream fout = new FileOutputStream( "writesomebytes.txt" );
FileChannel fcout = fout.getChannel();

while (true) {
  buffer.clear();
  int r = fcin.read( buffer );
  if (r==-1) {
    break;
  }
  buffer.flip();
  fcout.write( buffer );
}

Advanced topics for Buffer

1. Wrap an existing byte array into ByteBuffer

//allocate empty byte array to the Buffer for particular size
ByteBuffer buffer = ByteBuffer.allocate( 1024 );

//or you can turn an existing byte array into ByteBuffer thru wrap
byte array[] = new byte[1024];
ByteBuffer buffer = ByteBuffer.wrap( array );

You must be very careful about wrap() operation. Once you've done it, the underlying data can be accessed through the buffer as well as directly.

2. Buffer slicing

It creates a sub-buffer from an existing buffer

ByteBuffer buffer = ByteBuffer.allocate( 10 );
//fill it up
for (int i=0; i<buffer.capacity(); ++i) {
   buffer.put( (byte)i );
}
//slice a piece out (eg. slot 3 to 6). But the underlying data structure is shared
buffer.position( 3 );
buffer.limit( 7 );
ByteBuffer slice = buffer.slice();

for (int i=0; i<slice.capacity(); ++i) {
  byte b = slice.get( i );
  b *= 11;
  slice.put( i, b );
}

buffer.position( 0 );
buffer.limit( buffer.capacity() );
while (buffer.remaining()>0) {
   System.out.println( buffer.get() ); //you see the change as shared
}

$ java SliceBuffer
0
1
2
33
44
55
66
7
8
9

3. Read-ony Buffer

You can turn any regular buffer into a read-only buffer by calling its asReadOnlyBuffer() method, which returns a new buffer that is identical to the first (and shares data with it), but is read-only. On the other hand, you cannot convert a read-only buffer to a writable buffer. So, you can be for sure it will not be changed by others after you pass this around.

4. Direct and Indirect Buffer Another useful kind of ByteBuffer is the direct buffer. A direct buffer is one whose memory is allocated in a special way to increase I/O speed.

Actually, the exact definition of a direct buffer is implementation-dependent. Sun's documentation has this to say about direct buffers:

Given a direct byte buffer, the Java virtual machine will make a best effort to perform native I/O operations directly upon it. That is, it will attempt to avoid copying the buffer's content to (or from) an intermediate buffer before (or after) each invocation of one of the underlying operating system's native I/O operations.

You can see direct buffers in action in the example program FastCopyFile.java, which is a version of CopyFile.java that uses direct buffers for increased speed. You can also create a direct buffer using memory-mapped files.

Memory-mapped file I/O is a method for reading and writing file data that can be a great deal faster than regular stream- or channel-based I/O. Memory-mapped file I/O is accomplished by causing the data in a file to magically appear as the contents of a memory array. At first, this sounds like it simply means reading the entire file into memory, but in fact it does not. In general, only the parts of the file that you actually read or write are brought, or mapped, into memory.

Memory-mapping isn't really magical, or all that uncommon. Modern operating systems generally implement filesystems by mapping portions of a file into portions of memory, doing so on demand. The Java memory-mapping system simply provides access to this facility if it is available in the underlying operating system. Although they are fairly simple to create, writing to memory-mapped files can be dangerous. By the simple act of changing a single element of an array, you are directly modifying the file on disk. There is no separation between modifying the data and saving it to a disk.

//map a file channel to memory mapped buffer
MappedByteBuffer mbb = fc.map( FileChannel.MapMode.READ_WRITE, 0, 1024 );

Scattering read and gathering write

scattering read is like a regular channel read, except that it reads data into an array of buffers rather than a single buffer. The buffers are first inserted into an array, then the array passed as parameter to the channel.read() method. The read() method then writes data from the channel in the sequence the buffers occur in the array. Once a buffer is full, the channel moves on to fill the next buffer. The fact that scattering reads fill up one buffer before moving on to the next, means that it is not suited for dynamically sized message parts. In other words, if you have a header and a body, and the header is fixed size (e.g. 128 bytes), then a scattering read works fine.

ByteBuffer header = ByteBuffer.allocate(128);
ByteBuffer body   = ByteBuffer.allocate(1024);

ByteBuffer[] bufferArray = { header, body };

channel.read(bufferArray);

A gathering write writes data from multiple buffers into a single channel. Only the data between position and limit of the buffers is written. Thus, if a buffer has a capacity of 128 bytes, but only contains 58 bytes, only 58 bytes are written from that buffer to the channel. Thus, a gathering write works fine with dynamically sized message parts, in contrast to scattering reads.

ByteBuffer header = ByteBuffer.allocate(128);
ByteBuffer body   = ByteBuffer.allocate(1024);

//write data into buffers

ByteBuffer[] bufferArray = { header, body };

channel.write(bufferArray);

You can see scattering reads and gathering writes in action in the example program UseScatterGather.java.

Channel to Channel Transfer

The Java class libraries support zero copy on Linux and UNIX systems through the transferTo() method in java.nio.channels.FileChannel. You can use the transferTo() method to transfer bytes directly from the channel on which it is invoked to another writable byte channel. Or vice versa thru transferFrom(). That is to say, FileChannel is needed for data transfer across channels, without requiring data to flow through the application.

RandomAccessFile fromFile = new RandomAccessFile("fromFile.txt", "rw");
FileChannel      fromChannel = fromFile.getChannel();

RandomAccessFile toFile = new RandomAccessFile("toFile.txt", "rw");
FileChannel      toChannel = toFile.getChannel();

long position = 0;
long count    = fromChannel.size();

toChannel.transferFrom(fromChannel, position, count);
fromChannel.transferTo(position, count, toChannel);

Traditional approach

Copy data from file to socket

File.read(fileDesc, buf, len);
Socket.send(socket, buf, len);

Context switching

Zero copy

Not truly zero copy

True zero copy

We can further reduce the data duplication done by the kernel if the underlying network interface card supports gather operations. In Linux kernels 2.4 and later, the socket buffer descriptor was modified to accommodate this requirement. This approach not only reduces multiple context switches but also eliminates the duplicated data copies that require CPU involvement.

Context switching

Behind the scene, transferTo() from Channel will be routed to the sendfile() system call, which transfers data from one file descriptor to another:

#include <sys/socket.h>
ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);

In conclusion, zero copy reduces not only the CPU time spent in context switches between kernel and user space, but also the memory pressure in the JVM heap. It is by far the greatest trick provided by NIO I have seen.

ref: http://www.ibm.com/developerworks/library/j-zerocopy/

Selector and Channel

You can use just one thread to handle all of your channels via Selector. Switching between threads is expensive for an operating system, and each thread takes up some resources (memory) in the operating system too. Therefore, the less threads you use, the better. Keep in mind though, that modern operating systems and CPU's become better and better at multitasking, so the overheads of multithreading becomes smaller over time. In fact, if a CPU has multiple cores, you might be wasting CPU power by not multitasking.

//create selector
Selector selector = Selector.open();

//channel must be in non-blocking mode to be used with a Selector. 
//This means that you cannot use FileChannel's with a Selector
//since FileChannel's cannot be switched into non-blocking mode. 
//Socket channels will work fine though.
channel.configureBlocking(false);

//you must register the Channel with the Selector in order to use it
SelectionKey key = channel.register(selector, SelectionKey.OP_READ);

Notice the second parameter of the register() method. This is an "interest set", meaning what events you are interested in listening for in the Channel, via the Selector. There are four different events you can listen for:

• Connect - SelectionKey.OP_CONNECT
• Accept - SelectionKey.OP_ACCEPT
• Read - SelectionKey.OP_READ
• Write - SelectionKey.OP_WRITE

//if you are interested in more than one event
int interestSet = SelectionKey.OP_READ | SelectionKey.OP_WRITE;

Networking and Async IO

Asynchronous I/O is a method for reading and writing data without blocking. Normally, when your code makes a read() call, the code blocks until there is data to be read. Likewise, a write() call will block until the data can be written. Asynchronous I/O calls, on the other hand, do not block. Instead, you register your interest in a particular I/O event -- the arrival of readable data, a new socket connection, and so on -- and the system tells you when such an event occurs. With this, it lets you do I/O from a great many inputs and outputs at the same time. Synchronous programs often have to resort to polling, or to the creation of many, many threads, to deal with lots of connections. With asynchronous I/O, you can listen for I/O events on an arbitrary number of channels, without polling and without extra threads.

//Bind a non-blocking server socket channel to a port
ServerSocketChannel ssc = ServerSocketChannel.open();
ssc.configureBlocking( false );
ServerSocket ss = ssc.socket();
InetSocketAddress address = new InetSocketAddress( ports[i] );
ss.bind( address );

//Here specifies that we want to listen for accept events -- that is, the
//events that occur when a new connection is made. This is the only kind 
//of event that is appropriate for a ServerSocketChannel. A SelectionKey 
//represents this registration of this channel with this Selector. 
SelectionKey key = ssc.register( selector, SelectionKey.OP_ACCEPT );

When a Selector notifies you of an incoming event, it does this by supplying the SelectionKey that corresponds to that event.

//This method blocks until at least one of the registered events occurs
int num = selector.select();

//Returns a Set of the SelectionKey objects for which events have occurred.
Set selectedKeys = selector.selectedKeys();

Iterator it = selectedKeys.iterator();
while (it.hasNext()) {
   SelectionKey key = (SelectionKey)it.next();
   // ... deal with I/O event ...
}

NIO vs NIO.2

• java.nio.file package is added
• NIO a more abstract low level data I/O and NIO2 focused on file management.
• Java SE 7 introduces asynchronous IO with JSR203 making "true" proactor style IO handling implementations possible

Reference

• http://www.javaworld.com/article/2078654/java-se/java-se-five-ways-to-maximize-java-nio-and-nio-2.html
• https://github.com/jjenkov/java-nio-server
• http://howtodoinjava.com/2015/01/16/java-nio-2-0-memory-mapped-files-mappedbytebuffer-tutorial/
• http://cs.brown.edu/courses/cs161/papers/j-nio-ltr.pdf
• http://www.ibm.com/developerworks/library/j-zerocopy/
• http://www.javacodegeeks.com/2012/08/io-demystified.html