`module Reader`

This provides a read-only interface for data, which is surprisingly tricky to implement correctly.

Definitions

`def read(length = nil, buffer = nil)`

Read data from the underlying stream.

If given a non-negative length, it will read at most that many bytes from the stream. If the stream is at EOF, it will return nil.

If the length is not given, it will read all data until EOF, or return an empty string if the stream is already at EOF.

If buffer is given, then the read data will be placed into buffer instead of a newly created String object.

Signature

parameter　buffer　String: the buffer which will receive the data
returns　String: a buffer containing the data

Implementation

def read(length = nil, buffer = nil)
	return "" if length == 0
	
	buffer ||= String.new.force_encoding(Encoding::BINARY)
	
	# Take any previously buffered data and replace it into the given buffer.
	if @buffer
		buffer.replace(@buffer)
		@buffer = nil
	else
		buffer.clear
	end
	
	if length
		while buffer.bytesize < length and chunk = read_next
			buffer << chunk
		end
		
		# This ensures the subsequent `slice!` works correctly.
		buffer.force_encoding(Encoding::BINARY)
		
		# This will be at least one copy:
		@buffer = buffer.byteslice(length, buffer.bytesize)
		
		# This should be zero-copy:
		buffer.slice!(length, buffer.bytesize)
		
		if buffer.empty?
			return nil
		else
			return buffer
		end
	else
		while chunk = read_next
			buffer << chunk
		end
		
		return buffer
	end
end

`def read_partial(length = nil, buffer = nil)`

Read some bytes from the stream.

If the length is given, at most length bytes will be read. Otherwise, one chunk of data from the underlying stream will be read.

Will avoid reading from the underlying stream if there is buffered data available.

Signature

parameter　length　Integer: The maximum number of bytes to read.

Implementation

def read_partial(length = nil, buffer = nil)
	if @buffer
		if buffer
			buffer.replace(@buffer)
		else
			buffer = @buffer
		end
		@buffer = nil
	else
		if chunk = read_next
			if buffer
				buffer.replace(chunk)
			else
				buffer = chunk
			end
		else
			buffer&.clear
			buffer = nil
		end
	end
	
	if buffer and length
		if buffer.bytesize > length
			# This ensures the subsequent `slice!` works correctly.
			buffer.force_encoding(Encoding::BINARY)

			@buffer = buffer.byteslice(length, buffer.bytesize)
			buffer.slice!(length, buffer.bytesize)
		end
	end
	
	return buffer
end

`def readpartial(length, buffer = nil)`

Similar to Protocol::HTTP::Body::Stream::Reader#read_partial but raises an EOFError if the stream is at EOF.

Signature

parameter　length　Integer: The maximum number of bytes to read.
parameter　buffer　String: The buffer to read into.

Implementation

def readpartial(length, buffer = nil)
	read_partial(length, buffer) or raise EOFError, "End of file reached!"
end

`def each(&block)`

Iterate over each chunk of data from the input stream.

Signature

yields　{|chunk| ...}: Each chunk of data.

Implementation

def each(&block)
	return to_enum unless block_given?
	
	if @buffer
		yield @buffer
		@buffer = nil
	end
	
	while chunk = read_next
		yield chunk
	end
end

`def read_nonblock(length, buffer = nil, exception: nil)`

Read data from the stream without blocking if possible.

Signature

parameter　length　Integer: The maximum number of bytes to read.
parameter　buffer　String | Nil: The buffer to read into.

Implementation

def read_nonblock(length, buffer = nil, exception: nil)
	@buffer ||= read_next
	chunk = nil
	
	unless @buffer
		buffer&.clear
		return
	end
	
	if @buffer.bytesize > length
		chunk = @buffer.byteslice(0, length)
		@buffer = @buffer.byteslice(length, @buffer.bytesize)
	else
		chunk = @buffer
		@buffer = nil
	end
	
	if buffer
		buffer.replace(chunk)
	else
		buffer = chunk
	end
	
	return buffer
end

`def read_until(pattern, offset = 0, chomp: false)`

Read data from the stream until encountering pattern.

Signature

parameter　pattern　String: The pattern to match.
parameter　offset　Integer: The offset to start searching from.
parameter　chomp　Boolean: Whether to remove the pattern from the returned data.
returns　String: The contents of the stream up until the pattern, which is consumed but not returned.

Implementation

def read_until(pattern, offset = 0, chomp: false)
	# We don't want to split on the pattern, so we subtract the size of the pattern.
	split_offset = pattern.bytesize - 1
	
	@buffer ||= read_next
	return nil if @buffer.nil?
	
	until index = @buffer.index(pattern, offset)
		offset = @buffer.bytesize - split_offset
		
		offset = 0 if offset < 0
		
		if chunk = read_next
			@buffer << chunk
		else
			return nil
		end
	end
	
	@buffer.freeze
	matched = @buffer.byteslice(0, index+(chomp ? 0 : pattern.bytesize))
	@buffer = @buffer.byteslice(index+pattern.bytesize, @buffer.bytesize)
	
	return matched
end

`def gets(separator = NEWLINE, limit = nil, chomp: false)`

Read a single line from the stream.

Signature

parameter　separator　String: The line separator, defaults to \n.
parameter　limit　Integer: The maximum number of bytes to read.
parameter　*options　Hash: Additional options, passed to Protocol::HTTP::Body::Stream::Reader#read_until.

Implementation

def gets(separator = NEWLINE, limit = nil, chomp: false)
	# If the separator is an integer, it is actually the limit:
	if separator.is_a?(Integer)
		limit = separator
		separator = NEWLINE
	end
	
	# If no separator is given, this is the same as a read operation:
	if separator.nil?
		# I tried using `read(limit)` here but it will block until the limit is reached, which is not usually desirable behaviour.
		return read_partial(limit)
	end
	
	# We don't want to split on the separator, so we subtract the size of the separator:
	split_offset = separator.bytesize - 1
	
	@buffer ||= read_next
	return nil if @buffer.nil?
	
	offset = 0
	until index = @buffer.index(separator, offset)
		offset = @buffer.bytesize - split_offset
		offset = 0 if offset < 0
		
		# If we have gone past the limit, we are done:
		if limit and offset >= limit
			@buffer.freeze
			matched = @buffer.byteslice(0, limit)
			@buffer = @buffer.byteslice(limit, @buffer.bytesize)
			return matched
		end
		
		# Read more data:
		if chunk = read_next
			@buffer << chunk
		else
			# No more data could be read, return the remaining data:
			buffer = @buffer
			@buffer = nil
			
			return @buffer
		end
	end
	
	# Freeze the buffer, as this enables us to use byteslice without generating a hidden copy:
	@buffer.freeze
	
	if limit and index > limit
		line = @buffer.byteslice(0, limit)
		@buffer = @buffer.byteslice(limit, @buffer.bytesize)
	else
		line = @buffer.byteslice(0, index+(chomp ? 0 : separator.bytesize))
		@buffer = @buffer.byteslice(index+separator.bytesize, @buffer.bytesize)
	end
	
	return line
end