class Asciidoctor::Reader

Public: Methods for retrieving lines from AsciiDoc source files

Attributes

lineno[R]

Public: Get the 1-based offset of the current line.

source[R]

Public: Get the document source as a String Array of lines.

Public Class Methods

new(data = nil, document = nil, preprocess = false, &block) click to toggle source

Public: Initialize the Reader object.

data - The Array of Strings holding the Asciidoc source document. The

original instance of this Array is not modified (default: nil)

document - The document with which this reader is associated. Used to access

document attributes (default: nil)

preprocess - A flag indicating whether to run the preprocessor on these lines.

Only enable for the outer-most Reader. If this argument is true,
a Document object must also be supplied.
(default: false)

block - A block that can be used to retrieve external Asciidoc

data to include in this document.

Examples

data   = File.readlines(filename)
reader = Asciidoctor::Reader.new data
# File lib/asciidoctor/reader.rb, line 28
def initialize(data = nil, document = nil, preprocess = false, &block)
  data = [] if data.nil?
  # TODO use Struct to track file/lineno info; track as file changes; offset for sub-readers
  @lineno = 0
  if !preprocess
    @lines = data.is_a?(String) ? data.lines.entries : data.dup
    @preprocess_source = false
  elsif !data.empty?
    # NOTE we assume document is not nil!
    @document = document
    @preprocess_source = true
    @include_block = block_given? ? block : nil
    normalize_data(data.is_a?(String) ? data.lines.entries : data)
  else
    @lines = []
    @preprocess_source = false
  end

  @source = @lines.dup

  @next_line_preprocessed = false
  @unescape_next_line = false

  @conditionals_stack = []
  @skipping = false
  @eof = false
end

Public Instance Methods

advance() click to toggle source

Public: Advance to the next line by discarding the line at the front of the stack

Removes the line at the front of the stack without any processing.

returns a boolean indicating whether there was a line to discard

# File lib/asciidoctor/reader.rb, line 211
def advance
  @next_line_preprocessed = false
  # we assume that we're advancing over a line of known content
  if @eof || (@eof = @lines.empty?)
    false
  else
    @lineno += 1
    @lines.shift
    true
  end
end
chomp_last!() click to toggle source

Public: Chomp the String on the last line if this reader contains at least one line

Delegates to chomp!

Returns nil

# File lib/asciidoctor/reader.rb, line 483
def chomp_last!
  @lines.last.chomp! unless @eof || (@eof = @lines.empty?)
  nil
end
consume_comments(options = {}) click to toggle source

Public: Consume consecutive lines containing line- or block-level comments.

Returns the Array of lines that were consumed

Examples

@lines
=> ["// foo\n", "////\n", "foo bar\n", "////\n", "actual text\n"]

comment_lines = consume_comments
=> ["// foo\n", "////\n", "foo bar\n", "////\n"]

@lines
=> ["actual text\n"]
# File lib/asciidoctor/reader.rb, line 129
def consume_comments(options = {})
  comment_lines = []
  preprocess = options.fetch(:preprocess, true)
  while !(next_line = get_line(preprocess)).nil?
    if options[:include_blank_lines] && next_line.chomp.empty?
      comment_lines << next_line
    elsif (commentish = next_line.start_with?('//')) && (match = next_line.match(REGEXP[:comment_blk]))
      comment_lines << next_line
      comment_lines.push(*(grab_lines_until(:terminator => match[0], :grab_last_line => true, :preprocess => false)))
    elsif commentish && next_line.match(REGEXP[:comment])
      comment_lines << next_line
    else
      # throw it back
      unshift_line next_line
      break
    end
  end

  comment_lines
end
Also aliased as: skip_comment_lines
consume_line_comments() click to toggle source

Public: Consume consecutive lines containing line comments.

Returns the Array of lines that were consumed

Examples

@lines
=> ["// foo\n", "bar\n"]

comment_lines = consume_comments
=> ["// foo\n"]

@lines
=> ["bar\n"]
# File lib/asciidoctor/reader.rb, line 164
def consume_line_comments
  comment_lines = []
  # optimized code for shortest execution path
  while !(next_line = get_line).nil?
    if next_line.match(REGEXP[:comment])
      comment_lines << next_line
    else
      unshift_line next_line
      break
    end
  end

  comment_lines
end
empty?() click to toggle source

Public: Check whether this reader is empty (contains no lines)

If preprocessing is enabled for this Reader, and there are lines remaining, the next line is preprocessed before checking whether there are more lines.

Returns true if @lines is empty, otherwise false.

# File lib/asciidoctor/reader.rb, line 83
def empty?
  !has_more_lines?
end
get_line(preprocess = true) click to toggle source

Public: Get the next line of source data. Consumes the line returned.

preprocess - A Boolean flag indicating whether to evaluate preprocessing

directives (macros) before reading line (default: true)

Returns the String of the next line of the source data if data is present. Returns nil if there is no more data.

# File lib/asciidoctor/reader.rb, line 186
def get_line(preprocess = true)
  if @eof || (@eof = @lines.empty?)
    @next_line_preprocessed = true
    nil
  elsif preprocess && @preprocess_source &&
      !@next_line_preprocessed && preprocess_next_line.nil?
    @next_line_preprocessed = true
    nil
  else
    @lineno += 1
    @next_line_preprocessed = false
    if @unescape_next_line
      @unescape_next_line = false
      @lines.shift[1..-1]
    else
      @lines.shift
    end
  end
end
grab_lines_until(options = {}) { |this_line| ... } click to toggle source

Public: Return all the lines from `@lines` until we (1) run out them,

(2) find a blank line with :break_on_blank_lines => true, or (3) find
a line for which the given block evals to true.

options - an optional Hash of processing options:

* :break_on_blank_lines may be used to specify to break on
    blank lines
* :skip_first_line may be used to tell the reader to advance
    beyond the first line before beginning the scan
* :preserve_last_line may be used to specify that the String
    causing the method to stop processing lines should be
    pushed back onto the `lines` Array.
* :grab_last_line may be used to specify that the String
    causing the method to stop processing lines should be
    included in the lines being returned

Returns the Array of lines forming the next segment.

Examples

reader = Reader.new ["First paragraph\n", "Second paragraph\n",
                     "Open block\n", "\n", "Can have blank lines\n",
                     "--\n", "\n", "In a different segment\n"]

reader.grab_lines_until
=> ["First paragraph\n", "Second paragraph\n", "Open block\n"]
# File lib/asciidoctor/reader.rb, line 514
def grab_lines_until(options = {}, &block)
  buffer = []

  finis = false
  advance if options[:skip_first_line]
  # save options to locals for minor optimization
  terminator = options[:terminator]
  terminator.chomp! if terminator
  break_on_blank_lines = options[:break_on_blank_lines]
  break_on_list_continuation = options[:break_on_list_continuation]
  skip_line_comments = options[:skip_line_comments]
  preprocess = options.fetch(:preprocess, true)
  while !(this_line = get_line(preprocess)).nil?
    Debug.debug { "Reader processing line: '#{this_line}'" }
    finis = true if terminator && this_line.chomp == terminator
    finis = true if !finis && break_on_blank_lines && this_line.strip.empty?
    finis = true if !finis && break_on_list_continuation && this_line.chomp == LIST_CONTINUATION
    finis = true if !finis && block && yield(this_line)
    if finis
      buffer << this_line if options[:grab_last_line]
      unshift_line(this_line) if options[:preserve_last_line]
      break
    end

    unless skip_line_comments && this_line.match(REGEXP[:comment])
      buffer << this_line
    end
  end

  buffer
end
has_more_lines?() click to toggle source

Public: Check whether there are any lines left to read.

If preprocessing is enabled for this Reader, and there are lines remaining, the next line is preprocessed before checking whether there are more lines.

Returns true if @lines is empty, or false otherwise.

# File lib/asciidoctor/reader.rb, line 67
def has_more_lines?
  if @eof || (@eof = @lines.empty?)
    false
  elsif @preprocess_source && !@next_line_preprocessed
    preprocess_next_line.nil? ? false : !@lines.empty?
  else
    true
  end
end
lines() click to toggle source

Public: Get a copy of the remaining Array of String lines parsed from the source

# File lib/asciidoctor/reader.rb, line 57
def lines
  @lines.nil? ? nil : @lines.dup
end
normalize_data(data) click to toggle source

Private: Normalize raw input, used for the outermost Reader.

This method strips whitespace from the end of every line of the source data and appends a LF (i.e., Unix endline). This whitespace substitution is very important to how Asciidoctor works.

Any leading or trailing blank lines are also removed.

The normalized lines are assigned to the @lines instance variable.

data - A String Array of input data to be normalized

returns nothing

# File lib/asciidoctor/reader.rb, line 612
def normalize_data(data)
  # normalize line ending to LF (purging occurrences of CRLF)
  # this rstrip is *very* important to how Asciidoctor works
  @lines = data.map {|line| "#{line.rstrip}\n" }

  @lines.shift && @lineno += 1 while !@lines.first.nil? && @lines.first.chomp.empty?
  @lines.pop while !@lines.last.nil? && @lines.last.chomp.empty?

  # Process bibliography references, so they're available when text
  # before the reference is being rendered.
  # FIXME we don't have support for bibliography lists yet, so disable for now
  # plus, this should be done while we are walking lines above
  #@lines.each do |line|
  #  if biblio = line.match(REGEXP[:biblio])
  #    @document.register(:ids, biblio[1])
  #  end
  #end
  nil
end
peek_line(preprocess = true) click to toggle source

Public: Get the next line of source data. Does not consume the line returned.

preprocess - A Boolean flag indicating whether to evaluate preprocessing

directives (macros) before reading line (default: true)

Returns a String dup of the next line of the source data if data is present. Returns nil if there is no more data.

# File lib/asciidoctor/reader.rb, line 230
def peek_line(preprocess = true)
  if !preprocess
    # QUESTION do we need to dup?
    @eof || (@eof = @lines.empty?) ? nil : @lines.first.dup
  elsif has_more_lines?
    # QUESTION do we need to dup?
    @lines.first.dup
  else
    nil
  end
end
peek_lines(number = 1) click to toggle source

TODO document & test me!

# File lib/asciidoctor/reader.rb, line 243
def peek_lines(number = 1)
  lines = []
  idx = 0
  (1..number).each do
    if @preprocess_source && !@next_line_preprocessed
      advanced = preprocess_next_line
      break if advanced.nil? || @eof || (@eof = @lines.empty?)
      idx = 0 if advanced
    end
    break if idx >= @lines.size
    # QUESTION do we need to dup?
    lines << @lines[idx].dup
    idx += 1
  end
  lines
end
preprocess_conditional_inclusion(directive, target, delimiter, text) click to toggle source

Internal: Preprocess the directive (macro) to conditionally include content.

Preprocess the conditional inclusion directive (ifdef, ifndef, ifeval, endif) under the cursor. If the Reader is currently skipping content, then simply track the open and close delimiters of any nested conditional blocks. If the Reader is not skipping, mark whether the condition is satisfied and continue preprocessing recursively until the next line of available content is found.

directive - The conditional inclusion directive (ifdef, ifndef, ifeval, endif) target - The target, which is the name of one or more attributes that are

used in the condition (blank in the case of the ifeval directive)

delimiter - The conditional delimiter for multiple attributes ('+' means all

attributes must be defined or undefined, ',' means any of the attributes
can be defined or undefined.

text - The text associated with this directive (occurring between the square brackets)

Used for a single-line conditional block in the case of the ifdef or
ifndef directives, and for the conditional expression for the ifeval directive.

returns a Boolean indicating whether the cursor advanced, or nil if there are no more lines available.

# File lib/asciidoctor/reader.rb, line 319
def preprocess_conditional_inclusion(directive, target, delimiter, text)
  # must have a target before brackets if ifdef or ifndef
  # must not have text between brackets if endif
  # don't honor match if it doesn't meet this criteria
  if ((directive == 'ifdef' || directive == 'ifndef') && target.empty?) ||
      (directive == 'endif' && !text.nil?)
    @next_line_preprocessed = true
    return false
  end

  if directive == 'endif'
    stack_size = @conditionals_stack.size
    if stack_size > 0
      pair = @conditionals_stack.last
      if target.empty? || target == pair[:target]
        @conditionals_stack.pop
        @skipping = @conditionals_stack.empty? ? false : @conditionals_stack.last[:skipping]
      else
        puts "asciidoctor: ERROR: line #{@lineno + 1}: mismatched macro: endif::#{target}[], expected endif::#{pair[:target]}[]"
      end
    else
      puts "asciidoctor: ERROR: line #{@lineno + 1}: unmatched macro: endif::#{target}[]"
    end
    advance
    return preprocess_next_line.nil? ? nil : true
  end

  skip = nil
  if !@skipping
    case directive
    when 'ifdef'
      case delimiter
      when nil
        # if the attribute is undefined, then skip
        skip = !@document.attributes.has_key?(target)
      when ','
        # if any attribute is defined, then don't skip
        skip = !target.split(',').detect {|name| @document.attributes.has_key? name }
      when '+'
        # if any attribute is undefined, then skip
        skip = target.split('+').detect {|name| !@document.attributes.has_key? name }
      end
    when 'ifndef'
      case delimiter
      when nil
        # if the attribute is defined, then skip
        skip = @document.attributes.has_key?(target)
      when ','
        # if any attribute is undefined, then don't skip
        skip = !target.split(',').detect {|name| !@document.attributes.has_key? name }
      when '+'
        # if any attribute is defined, then skip
        skip = target.split('+').detect {|name| @document.attributes.has_key? name }
      end
    when 'ifeval'
      # the text in brackets must match an expression
      # don't honor match if it doesn't meet this criteria
      if !target.empty? || !(expr_match = text.strip.match(REGEXP[:eval_expr]))
        @next_line_preprocessed = true
        return false
      end

      lhs = resolve_expr_val(expr_match[1])
      op = expr_match[2]
      rhs = resolve_expr_val(expr_match[3])

      skip = !lhs.send(op.to_sym, rhs)
    end
    @skipping = skip
  end
  advance
  # single line conditional inclusion
  if directive != 'ifeval' && !text.nil?
    if !@skipping
      unshift_line "#{text.rstrip}\n"
      return true
    end
  # conditional inclusion block
  else
    @conditionals_stack << {:target => target, :skip => skip, :skipping => @skipping}
  end
  return preprocess_next_line.nil? ? nil : true
end
preprocess_include(target) click to toggle source

Internal: Preprocess the directive (macro) to include the target document.

Preprocess the directive to include the target document. The scenarios are as follows:

If SafeMode is SECURE or greater, the directive is ignore and the include directive line is emitted verbatim.

Otherwise, if an include handler is specified (currently controlled by a closure block), pass the target to that block and expect an Array of String lines in return.

Otherwise, if the include-depth attribute is greater than 0, normalize the target path and read the lines onto the beginning of the Array of source data.

If none of the above apply, emit the include directive line verbatim.

target - The name of the source document to include as specified in the

target slot of the include::[] macro

returns a Boolean indicating whether the line under the cursor has changed.

# File lib/asciidoctor/reader.rb, line 425
def preprocess_include(target)
  # if running in SafeMode::SECURE or greater, don't process this directive
  if @document.safe >= SafeMode::SECURE
    @next_line_preprocessed = true
    false
  # assume that if a block is given, the developer wants
  # to handle when and how to process the include, even
  # if the include-depth attribute is 0
  elsif @include_block
    advance
    # FIXME this borks line numbers
    @lines.unshift(*@include_block.call(target).map {|l| "#{l.rstrip}\n"})
  # FIXME currently we're not checking the upper bound of the include depth
  elsif @document.attributes.fetch('include-depth', 0).to_i > 0
    advance
    # FIXME this borks line numbers
    @lines.unshift(*File.readlines(@document.normalize_asset_path(target, 'include file')).map {|l| "#{l.rstrip}\n"})
    true
  else
    @next_line_preprocessed = true
    false
  end
end
preprocess_next_line() click to toggle source

Internal: Preprocess the next line until the cursor is at a line of content

Evaluate preprocessor macros on the next line, continuing to do so until the cursor arrives at a line of included content. That line is marked as preprocessed so that preprocessing is not performed multiple times.

returns a Boolean indicating whether the cursor advanced, or nil if there are no more lines available.

# File lib/asciidoctor/reader.rb, line 268
def preprocess_next_line
  # this return could be happening from a recursive call
  return nil if @eof || (next_line = @lines.first).nil?
  if next_line.include?('if') && (match = next_line.match(REGEXP[:ifdef_macro]))
    if next_line.start_with? '\'
      @next_line_preprocessed = true
      @unescape_next_line = true
      false
    else
      preprocess_conditional_inclusion(*match.captures)
    end
  elsif @skipping
    advance
    # skip over comment blocks, we don't want to process directives in them
    skip_comment_lines :include_blank_lines => true, :preprocess => false
    preprocess_next_line.nil? ? nil : true
  elsif next_line.include?('include::') && match = next_line.match(REGEXP[:include_macro])
    if next_line.start_with? '\'
      @next_line_preprocessed = true
      @unescape_next_line = true
      false
    else
      preprocess_include(match[1])
    end
  else
    @next_line_preprocessed = true
    false
  end
end
resolve_expr_val(str) click to toggle source

Private: Resolve the value of one side of the expression

# File lib/asciidoctor/reader.rb, line 567
def resolve_expr_val(str)
  val = str
  type = nil

  if val.start_with?('"') && val.end_with?('"') ||
      val.start_with?('\') && val.end_with?('\')
    type = :s
    val = val[1..-2]
  end

  if val.include? '{'
    val = @document.sub_attributes(val)
  end

  if type != :s
    if val.empty?
      val = nil
    elsif val == 'true'
      val = true
    elsif val == 'false'
      val = false
    elsif val.include?('.')
      val = val.to_f
    else
      val = val.to_i
    end
  end

  val
end
sanitize_attribute_name(name) click to toggle source

Public: Convert a string to a legal attribute name.

name - The String holding the Asciidoc attribute name.

Returns a String with the legal name.

Examples

sanitize_attribute_name('Foo Bar')
=> 'foobar'

sanitize_attribute_name('foo')
=> 'foo'

sanitize_attribute_name('Foo 3 #-Billy')
=> 'foo3-billy'
# File lib/asciidoctor/reader.rb, line 562
def sanitize_attribute_name(name)
  Lexer.sanitize_attribute_name(name)
end
skip_blank_lines() click to toggle source

Private: Strip off leading blank lines in the Array of lines.

Examples

@lines
=> ["\n", "\t\n", "Foo\n", "Bar\n", "\n"]

skip_blank_lines
=> 2

@lines
=> ["Foo\n", "Bar\n"]

Returns an Integer of the number of lines skipped

# File lib/asciidoctor/reader.rb, line 101
def skip_blank_lines
  skipped = 0
  # optimized code for shortest execution path
  while !(next_line = get_line).nil?
    if next_line.chomp.empty?
      skipped += 1
    else
      unshift_line next_line
      break
    end
  end 

  skipped
end
skip_comment_lines(options = {}) click to toggle source
Alias for: consume_comments
unshift(*new_lines) click to toggle source

Public: Push Array of lines onto the front of the Array of source data, unless `lines` has no non-nil values.

Returns nil

# File lib/asciidoctor/reader.rb, line 466
def unshift(*new_lines)
  size = new_lines.size
  if size > 0
    @lines.unshift(*new_lines)
    # assume that what we are putting back on is already processed for directives
    @next_line_preprocessed = true
    @eof = false
    @lineno -= size
  end
  nil
end
unshift_line(line) click to toggle source

Public: Push the String line onto the beginning of the Array of source data.

Since this line was (assumed to be) previously retrieved through the reader, it is marked as preprocessed.

returns nil

# File lib/asciidoctor/reader.rb, line 455
def unshift_line(line)
  @lines.unshift line
  @next_line_preprocessed = true
  @eof = false
  @lineno -= 1
  nil
end