`module Readable`

A module providing readable stream functionality.

You must implement the sysread method to read data from the underlying IO. You may implement sysread_nonblock to support non-blocking partial peeks.

Definitions

`def self.async_safe?(method)`

Check if a method is async-safe.

Signature

parameter　method　Symbol: The method name to check.
returns　Symbol | Boolean: The concurrency guard for the given method.

Implementation

def self.async_safe?(method)
	ASYNC_SAFE.fetch(method, false)
end

`def initialize(minimum_read_size: MINIMUM_READ_SIZE, maximum_read_size: MAXIMUM_READ_SIZE, block_size: nil, **, &block)`

Initialize readable stream functionality.

Signature

parameter　minimum_read_size　Integer: The minimum size for read operations.
parameter　maximum_read_size　Integer: The maximum size for read operations.
parameter　block_size　Integer: Legacy parameter, use minimum_read_size instead.

Implementation

def initialize(minimum_read_size: MINIMUM_READ_SIZE, maximum_read_size: MAXIMUM_READ_SIZE, block_size: nil, **, &block)
	@finished = false
	@read_buffer = StringBuffer.new
	# Used as destination buffer for underlying reads.
	@input_buffer = StringBuffer.new
	
	# Support legacy block_size parameter for backwards compatibility
	@minimum_read_size = block_size || minimum_read_size
	@maximum_read_size = maximum_read_size
	
	super(**, &block) if defined?(super)
end

`def block_size`

Legacy accessor for backwards compatibility

Signature

returns　Integer: The minimum read size.

Implementation

def block_size
	@minimum_read_size
end

`def block_size=(value)`

Legacy setter for backwards compatibility

Signature

parameter　value　Integer: The minimum read size.

Implementation

def block_size=(value)
	@minimum_read_size = value
end

`def read(size = nil, buffer = nil)`

Read data from the stream.

Signature

parameter　size　Integer | Nil: The number of bytes to read. If nil, read until end of stream.
parameter　buffer　String | Nil: An optional buffer to fill with data instead of allocating a new string.
returns　String: The data read from the stream, or the provided buffer filled with data.

Implementation

def read(size = nil, buffer = nil)
	if size == 0
		if buffer
			buffer.clear
			buffer.force_encoding(Encoding::BINARY)
			return buffer
		else
			return String.new(encoding: Encoding::BINARY)
		end
	end
	
	if size
		until @finished or @read_buffer.bytesize >= size
			# Compute the amount of data we need to read from the underlying stream:
			read_size = size - @read_buffer.bytesize
			
			# Don't read less than @minimum_read_size to avoid lots of small reads:
			fill_read_buffer(read_size > @minimum_read_size ? read_size : @minimum_read_size)
		end
	else
		until @finished
			fill_read_buffer
		end
		
		if buffer
			buffer.replace(@read_buffer)
			@read_buffer.clear
		else
			buffer = @read_buffer
			@read_buffer = StringBuffer.new
		end
		
		# Read without size always returns a non-nil value, even if it is an empty string.
		return buffer
	end
	
	return consume_read_buffer(size, buffer)
end

`def read_partial(size = nil, buffer = nil)`

Read at most size bytes from the stream. Will avoid reading from the underlying stream if possible.

Signature

parameter　size　Integer | Nil: The number of bytes to read. If nil, read all available data.
parameter　buffer　String | Nil: An optional buffer to fill with data instead of allocating a new string.
returns　String: The data read from the stream, or the provided buffer filled with data.

Implementation

def read_partial(size = nil, buffer = nil)
	if size == 0
		if buffer
			buffer.clear
			buffer.force_encoding(Encoding::BINARY)
			return buffer
		else
			return String.new(encoding: Encoding::BINARY)
		end
	end
	
	if !@finished and @read_buffer.empty?
		fill_read_buffer
	end
	
	return consume_read_buffer(size, buffer)
end

`def read_exactly(size, buffer = nil, exception: EOFError)`

Read exactly the specified number of bytes.

Signature

parameter　size　Integer: The number of bytes to read.
parameter　exception　Class: The exception to raise if not enough data is available.
returns　String: The data read from the stream.

Implementation

def read_exactly(size, buffer = nil, exception: EOFError)
	if buffer = read(size, buffer)
		if buffer.bytesize != size
			raise exception, "Could not read enough data!"
		end
		
		return buffer
	end
	
	raise exception, "Stream finished before reading enough data!"
end

`def readpartial(size = nil, buffer = nil)`

This is a compatibility shim for existing code that uses readpartial.

Signature

parameter　size　Integer | Nil: The number of bytes to read.
parameter　buffer　String | Nil: An optional buffer to fill with data instead of allocating a new string.
returns　String: The data read from the stream.

Implementation

def readpartial(size = nil, buffer = nil)
	read_partial(size, buffer) or raise EOFError, "Stream finished before reading enough data!"
end

`def read_until(pattern, offset = 0, limit: nil, chomp: true)`

Efficiently read data from the stream until encountering pattern.

Signature

parameter　pattern　String: The pattern to match.
parameter　offset　Integer: The offset to start searching from.
parameter　limit　Integer: The maximum number of bytes to read, including the pattern (even if chomped).
parameter　chomp　Boolean: Whether to remove the pattern from the returned data.
returns　String | Nil: The contents of the stream up until the pattern, or nil if the pattern was not found.

Implementation

def read_until(pattern, offset = 0, limit: nil, chomp: true)
	if index = index_of(pattern, offset, limit)
		return nil if limit and index >= limit
		
		@read_buffer.freeze
		matched = @read_buffer.byteslice(0, index+(chomp ? 0 : pattern.bytesize))
		@read_buffer = @read_buffer.byteslice(index+pattern.bytesize, @read_buffer.bytesize)
		
		return matched
	end
end

`def discard_until(pattern, offset = 0, limit: nil)`

Efficiently discard data from the stream until encountering pattern.

Signature

parameter　pattern　String: The pattern to match.
parameter　offset　Integer: The offset to start searching from.
parameter　limit　Integer: The maximum number of bytes to read, including the pattern.
returns　String | Nil: The contents of the stream up until the pattern, or nil if the pattern was not found.

Implementation

def discard_until(pattern, offset = 0, limit: nil)
	if index = index_of(pattern, offset, limit, true)
		@read_buffer.freeze
		
		if limit and index >= limit
			@read_buffer = @read_buffer.byteslice(limit, @read_buffer.bytesize)
			
			return nil
		end
		
		matched = @read_buffer.byteslice(0, index+pattern.bytesize)
		@read_buffer = @read_buffer.byteslice(index+pattern.bytesize, @read_buffer.bytesize)
		
		return matched
	end
end

`def peek(size = nil)`

Peek at data in the buffer without consuming it.

Signature

parameter　size　Integer | Nil: The number of bytes to peek at. If nil, peek at all available data.
returns　String: The data in the buffer without consuming it.

Implementation

def peek(size = nil)
	if size == 0
		return String.new(encoding: Encoding::BINARY)
	end
	
	if size
		until @finished or @read_buffer.bytesize >= size
			# Compute the amount of data we need to read from the underlying stream:
			read_size = size - @read_buffer.bytesize
			
			# Don't read less than @minimum_read_size to avoid lots of small reads:
			fill_read_buffer(read_size > @minimum_read_size ? read_size : @minimum_read_size)
		end
		
		return @read_buffer[..([size, @read_buffer.size].min - 1)]
	end
	
	until (block_given? && yield(@read_buffer)) or @finished
		fill_read_buffer
	end
	
	return @read_buffer
end

`def peek_partial(size = @minimum_read_size)`

Peek at data without consuming it, making at most one non-blocking read attempt.

Any data read from the underlying stream is preserved in the read buffer. If the read would block or the stream is at EOF, this method returns nil.

After this method returns nil, IO::Stream::Readable#readable? indicates whether the read would block or EOF was observed.

Signature

parameter　size　Integer: The maximum number of bytes to peek at.
returns　String | Nil: The immediately available data, or nil if no data can be read without blocking.

Implementation

def peek_partial(size = @minimum_read_size)
	if size == 0
		return String.new(encoding: Encoding::BINARY)
	end
	
	if @read_buffer.empty?
		if @finished
			return nil
		end
		
		read_size = [size, @maximum_read_size].min
		
		result = sysread_nonblock(read_size, @read_buffer)
		case result
		when :wait_readable, :wait_writable
			return nil
		when nil
			@finished = true
			return nil
		end
	end
	
	return @read_buffer.byteslice(0, [size, @read_buffer.bytesize].min)
end

`def gets(separator = $/, limit = nil, chomp: false)`

Read a line from the stream, similar to IO#gets.

Signature

parameter　separator　String: The line separator to search for.
parameter　limit　Integer | Nil: The maximum number of bytes to read.
parameter　chomp　Boolean: Whether to remove the separator from the returned line.
returns　String | Nil: The line read from the stream, or nil if at end of stream.

Implementation

def gets(separator = $/, limit = nil, chomp: false)
	# Compatibility with IO#gets:
	if separator.is_a?(Integer)
		limit = separator
		separator = $/
	end
	
	# We don't want to split in the middle of the separator, so we subtract the size of the separator from the start of the search:
	split_offset = separator.bytesize - 1
	
	offset = 0
	
	until index = @read_buffer.index(separator, offset)
		offset = @read_buffer.bytesize - split_offset
		offset = 0 if offset < 0
		
		# If a limit was given, and the offset is beyond the limit, we should return up to the limit:
		if limit and offset >= limit
			# As we didn't find the separator, there is nothing to chomp either.
			return consume_read_buffer(limit)
		end
		
		# If we can't read any more data, we should return what we have:
		return consume_read_buffer unless fill_read_buffer
	end
	
	# If the index of the separator was beyond the limit:
	if limit and index >= limit
		# Return up to the limit:
		return consume_read_buffer(limit)
	end
	
	# Freeze the read buffer, as this enables us to use byteslice without generating a hidden copy:
	@read_buffer.freeze
	
	line = @read_buffer.byteslice(0, index+(chomp ? 0 : separator.bytesize))
	@read_buffer = @read_buffer.byteslice(index+separator.bytesize, @read_buffer.bytesize)
	
	return line
end

`def finished?`

Determins if the stream has consumed all available data. May block if the stream is not readable. See IO::Stream::Readable#readable? for a non-blocking alternative.

Signature

returns　Boolean: If the stream is at file which means there is no more data to be read.

Implementation

def finished?
	if !@read_buffer.empty?
		return false
	elsif @finished
		return true
	else
		return !self.fill_read_buffer
	end
end

`def finish!`

Mark the stream as finished and raise EOFError.

Implementation

def finish!
	@read_buffer.clear
	@finished = true
	
	raise EOFError
end

`def readable?`

Whether there is a chance that a read operation will succeed or not.

Signature

returns　Boolean: If the stream is readable, i.e. a read operation has a chance of success.

Implementation

def readable?
	# If we are at the end of the file, we can't read any more data:
	if @finished
		return false
	end
	
	# If the read buffer is not empty, we can read more data:
	if !@read_buffer.empty?
		return true
	end
	
	# If the underlying stream is readable, we can read more data:
	return !closed?
end

`def close_read`

Close the read end of the stream.

Implementation

def close_read
end

`def sysread_nonblock(size, buffer)`

Attempts to read data from the underlying stream without blocking. Implementations may override this method when non-blocking reads are supported.

Implementation

def sysread_nonblock(size, buffer)
	return :wait_readable
end

`def fill_read_buffer(size = @minimum_read_size)`

Fills the buffer from the underlying stream.

Implementation

def fill_read_buffer(size = @minimum_read_size)
	# Limit the read size to avoid exceeding SSIZE_MAX and to manage memory usage.
	# Very large reads can also hurt interactive performance by blocking for too long.
	if size > @maximum_read_size
		size = @maximum_read_size
	end
	
	# This effectively ties the input and output stream together.
	self.flush
	
	if @read_buffer.empty?
		if sysread(size, @read_buffer)
			# Console.info(self, name: "read") {@read_buffer.inspect}
			return true
		end
	else
		if chunk = sysread(size, @input_buffer)
			@read_buffer << chunk
			# Console.info(self, name: "read") {@read_buffer.inspect}
			
			return true
		end
	end
	
	# else for both cases above:
	@finished = true
	return false
end

`def consume_read_buffer(size = nil, buffer = nil)`

Consumes at most size bytes from the buffer.

Signature

parameter　size　Integer | Nil: The amount of data to consume. If nil, consume entire buffer.
parameter　buffer　String | Nil: An optional buffer to fill with data instead of allocating a new string.
returns　String | Nil: The consumed data, or nil if no data available.

Implementation

def consume_read_buffer(size = nil, buffer = nil)
	# If we are at finished, and the read buffer is empty, we can't consume anything.
	if @finished && @read_buffer.empty?
		# Clear the buffer even when returning nil
		if buffer
			buffer.clear
			buffer.force_encoding(Encoding::BINARY)
		end
		return nil
	end
	
	result = nil
	
	if size.nil? or size >= @read_buffer.bytesize
		# Consume the entire read buffer:
		if buffer
			buffer.clear
			buffer << @read_buffer
			result = buffer
		else
			result = @read_buffer
		end
		@read_buffer = StringBuffer.new
	else
		# We know that we are not going to reuse the original buffer.
		# But byteslice will generate a hidden copy. So let's freeze it first:
		@read_buffer.freeze
		
		if buffer
			# Use replace instead of clear + << for better performance
			buffer.replace(@read_buffer.byteslice(0, size))
			result = buffer
		else
			result = @read_buffer.byteslice(0, size)
		end
		@read_buffer = @read_buffer.byteslice(size, @read_buffer.bytesize)
	end
	
	return result
end