class Enumerator - Documentation for Ruby 3.2

# class Enumerator

A class which allows both internal and external iteration.

An `Enumerator` can be created by the following methods.

- `Object#to_enum`

- `Object#enum_for`

- `Enumerator.new`

Most methods have two forms: a block form where the contents are evaluated for each item in the enumeration, and a non-block form which returns a new `Enumerator` wrapping the iteration.

    enumerator = %w(one two three).each
    puts enumerator.class # => Enumerator

    enumerator.each_with_object("foo") do |item, obj|
      puts "#{obj}: #{item}"
    end

    # foo: one
    # foo: two
    # foo: three

    enum_with_obj = enumerator.each_with_object("foo")
    puts enum_with_obj.class # => Enumerator

    enum_with_obj.each do |item, obj|
      puts "#{obj}: #{item}"
    end

    # foo: one
    # foo: two
    # foo: three

This allows you to chain Enumerators together. For example, you can map a list’s elements to strings containing the index and the element as a string via:

     puts %w[foo bar baz].map.with_index { |w, i| "#{i}:#{w}" }
     # => ["0:foo", "1:bar", "2:baz"]

    == External Iteration

An `Enumerator` can also be used as an external iterator. For example, `Enumerator#next` returns the next value of the iterator or raises `StopIteration` if the `Enumerator` is at the end.

    e = [1,2,3].each   # returns an enumerator object.
    puts e.next   # => 1
    puts e.next   # => 2
    puts e.next   # => 3
    puts e.next   # raises StopIteration

`next`, `next_values`, `peek` and `peek_values` are the only methods which use external iteration (and `Array#zip(Enumerable-not-Array)` which uses `next`).

These methods do not affect other internal enumeration methods, unless the underlying iteration method itself has side-effect, e.g. `IO#each_line`.

External iteration differs significantly from internal iteration due to using a Fiber:

    - The Fiber adds some overhead compared to internal enumeration.
    - The stacktrace will only include the stack from the Enumerator, not above.
    - Fiber-local variables are *not* inherited inside the Enumerator Fiber,
      which instead starts with no Fiber-local variables.
    - Fiber storage variables *are* inherited and are designed
      to handle Enumerator Fibers. Assigning to a Fiber storage variable
      only affects the current Fiber, so if you want to change state
      in the caller Fiber of the Enumerator Fiber, you need to use an
      extra indirection (e.g., use some object in the Fiber storage
      variable and mutate some ivar of it).

Concretely:

     Thread.current[:fiber_local] = 1
     Fiber[:storage_var] = 1
     e = Enumerator.new do |y|
       p Thread.current[:fiber_local] # for external iteration: nil, for internal iteration: 1
       p Fiber[:storage_var] # => 1, inherited
       Fiber[:storage_var] += 1
       y << 42
     end

     p e.next # => 42
     p Fiber[:storage_var] # => 1 (it ran in a different Fiber)

     e.each { p _1 }
     p Fiber[:storage_var] # => 2 (it ran in the same Fiber/"stack" as the current Fiber)

    == Convert External Iteration to Internal Iteration

You can use an external iterator to implement an internal iterator as follows:

    def ext_each(e)
      while true
        begin
          vs = e.next_values
        rescue StopIteration
          return $!.result
        end
        y = yield(*vs)
        e.feed y
      end
    end

    o = Object.new

    def o.each
      puts yield
      puts yield(1)
      puts yield(1, 2)
      3
    end

    # use o.each as an internal iterator directly.
    puts o.each {|*x| puts x; [:b, *x] }
    # => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3

    # convert o.each to an external iterator for
    # implementing an internal iterator.
    puts ext_each(o.to_enum) {|*x| puts x; [:b, *x] }
    # => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3

### Public Class Methods

new(size = nil) { |yielder| ... } click to toggle source

Creates a new `Enumerator` object, which can be used as an `Enumerable`.

Iteration is defined by the given block, in which a “yielder” object, given as block parameter, can be used to yield a value by calling the `yield` method (aliased as `<<`):

    fib = Enumerator.new do |y|
      a = b = 1
      loop do
        y << a
        a, b = b, a + b
      end
    end

    fib.take(10) # => [1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

The optional parameter can be used to specify how to calculate the size in a lazy fashion (see `Enumerator#size`). It can either be a value or a callable object.

    static VALUE
    enumerator_initialize(int argc, VALUE *argv, VALUE obj)
    {
        VALUE iter = rb_block_proc();
        VALUE recv = generator_init(generator_allocate(rb_cGenerator), iter);
        VALUE arg0 = rb_check_arity(argc, 0, 1) ? argv[0] : Qnil;
        VALUE size = convert_to_feasible_size_value(arg0);

        return enumerator_init(obj, recv, sym_each, 0, 0, 0, size, false);
    }

produce(initial = nil) { |prev| block } → enumerator click to toggle source

Creates an infinite enumerator from any block, just called over and over. The result of the previous iteration is passed to the next one. If `initial` is provided, it is passed to the first iteration, and becomes the first element of the enumerator; if it is not provided, the first iteration receives `nil`, and its result becomes the first element of the iterator.

Raising `StopIteration` from the block stops an iteration.

    Enumerator.produce(1, &:succ)   # => enumerator of 1, 2, 3, 4, ....

    Enumerator.produce { rand(10) } # => infinite random number sequence

    ancestors = Enumerator.produce(node) { |prev| node = prev.parent or raise StopIteration }
    enclosing_section = ancestors.find { |n| n.type == :section }

Using `::produce` together with `Enumerable` methods like `Enumerable#detect`, `Enumerable#slice_after`, `Enumerable#take_while` can provide Enumerator-based alternatives for `while` and `until` cycles:

    # Find next Tuesday
    require "date"
    Enumerator.produce(Date.today, &:succ).detect(&:tuesday?)

    # Simple lexer:
    require "strscan"
    scanner = StringScanner.new("7+38/6")
    PATTERN = %r{\d+|[-/+*]}
    Enumerator.produce { scanner.scan(PATTERN) }.slice_after { scanner.eos? }.first
    # => ["7", "+", "38", "/", "6"]

    static VALUE
    enumerator_s_produce(int argc, VALUE *argv, VALUE klass)
    {
        VALUE init, producer;

        if (!rb_block_given_p()) rb_raise(rb_eArgError, "no block given");

        if (rb_scan_args(argc, argv, "01", &init) == 0) {
            init = Qundef;
        }

        producer = producer_init(producer_allocate(rb_cEnumProducer), init, rb_block_proc());

        return rb_enumeratorize_with_size_kw(producer, sym_each, 0, 0, producer_size, RB_NO_KEYWORDS);
    }

product(*enums) → enumerator click to toggle source
product(*enums) { |elts| ... } → enumerator

Generates a new enumerator object that generates a Cartesian product of given enumerable objects. This is equivalent to `Enumerator::Product.new`.

    e = Enumerator.product(1..3, [4, 5])
    e.to_a #=> [[1, 4], [1, 5], [2, 4], [2, 5], [3, 4], [3, 5]]
    e.size #=> 6

When a block is given, calls the block with each N-element array generated and returns `nil`.

    static VALUE
    enumerator_s_product(int argc, VALUE *argv, VALUE klass)
    {
        VALUE enums = Qnil, options = Qnil, block = Qnil;

        rb_scan_args(argc, argv, "*:&", &enums, &options, &block);

        if (!NIL_P(options) && !RHASH_EMPTY_P(options)) {
            rb_exc_raise(rb_keyword_error_new("unknown", rb_hash_keys(options)));
        }

        VALUE obj = enum_product_initialize(argc, argv, enum_product_allocate(rb_cEnumProduct));

        if (!NIL_P(block)) {
            enum_product_run(obj, block);
            return Qnil;
        }

        return obj;
    }

### Public Instance Methods

e + enum → enumerator click to toggle source

Returns an enumerator object generated from this enumerator and a given enumerable.

    e = (1..3).each + [4, 5]
    e.to_a #=> [1, 2, 3, 4, 5]

    static VALUE
    enumerator_plus(VALUE obj, VALUE eobj)
    {
        return new_enum_chain(rb_ary_new_from_args(2, obj, eobj));
    }

each { |elm| block } → obj click to toggle source
each → enum
each(*appending_args) { |elm| block } → obj
each(*appending_args) → an_enumerator

Iterates over the block according to how this `Enumerator` was constructed. If no block and no arguments are given, returns self.

### Examples¶ ↑

    "Hello, world!".scan(/\w+/)                     #=> ["Hello", "world"]
    "Hello, world!".to_enum(:scan, /\w+/).to_a      #=> ["Hello", "world"]
    "Hello, world!".to_enum(:scan).each(/\w+/).to_a #=> ["Hello", "world"]

    obj = Object.new

    def obj.each_arg(a, b=:b, *rest)
      yield a
      yield b
      yield rest
      :method_returned
    end

    enum = obj.to_enum :each_arg, :a, :x

    enum.each.to_a                  #=> [:a, :x, []]
    enum.each.equal?(enum)          #=> true
    enum.each { |elm| elm }         #=> :method_returned

    enum.each(:y, :z).to_a          #=> [:a, :x, [:y, :z]]
    enum.each(:y, :z).equal?(enum)  #=> false
    enum.each(:y, :z) { |elm| elm } #=> :method_returned

    static VALUE
    enumerator_each(int argc, VALUE *argv, VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);

        if (argc > 0) {
            VALUE args = (e = enumerator_ptr(obj = rb_obj_dup(obj)))->args;
            if (args) {
    #if SIZEOF_INT < SIZEOF_LONG
                /* check int range overflow */
                rb_long2int(RARRAY_LEN(args) + argc);
    #endif
                args = rb_ary_dup(args);
                rb_ary_cat(args, argv, argc);
            }
            else {
                args = rb_ary_new4(argc, argv);
            }
            e->args = args;
            e->size = Qnil;
            e->size_fn = 0;
        }
        if (!rb_block_given_p()) return obj;

        if (!lazy_precheck(e->procs)) return Qnil;

        return enumerator_block_call(obj, 0, obj);
    }

each_with_index {|(*args), idx| ... } click to toggle source
each_with_index

Same as `Enumerator#with_index(0)`, i.e. there is no starting offset.

If no block is given, a new `Enumerator` is returned that includes the index.

    static VALUE
    enumerator_each_with_index(VALUE obj)
    {
        return enumerator_with_index(0, NULL, obj);
    }

each_with_object(obj) {|(*args), obj| ... } click to toggle source
each_with_object(obj)

Iterates the given block for each element with an arbitrary object, `obj`, and returns `obj`

If no block is given, returns a new `Enumerator`.

### Example¶ ↑

    to_three = Enumerator.new do |y|
      3.times do |x|
        y << x
      end
    end

    to_three_with_string = to_three.with_object("foo")
    to_three_with_string.each do |x,string|
      puts "#{string}: #{x}"
    end

    # => foo: 0
    # => foo: 1
    # => foo: 2

    static VALUE
    enumerator_with_object(VALUE obj, VALUE memo)
    {
        RETURN_SIZED_ENUMERATOR(obj, 1, &memo, enumerator_enum_size);
        enumerator_block_call(obj, enumerator_with_object_i, memo);

        return memo;
    }

Also aliased as: with_object
feed obj → nil click to toggle source

Sets the value to be returned by the next yield inside `e`.

If the value is not set, the yield returns nil.

This value is cleared after being yielded.

    # Array#map passes the array's elements to "yield" and collects the
    # results of "yield" as an array.
    # Following example shows that "next" returns the passed elements and
    # values passed to "feed" are collected as an array which can be
    # obtained by StopIteration#result.
    e = [1,2,3].map
    p e.next           #=> 1
    e.feed "a"
    p e.next           #=> 2
    e.feed "b"
    p e.next           #=> 3
    e.feed "c"
    begin
      e.next
    rescue StopIteration
      p $!.result      #=> ["a", "b", "c"]
    end

    o = Object.new
    def o.each
      x = yield         # (2) blocks
      p x               # (5) => "foo"
      x = yield         # (6) blocks
      p x               # (8) => nil
      x = yield         # (9) blocks
      p x               # not reached w/o another e.next
    end

    e = o.to_enum
    e.next              # (1)
    e.feed "foo"        # (3)
    e.next              # (4)
    e.next              # (7)
                        # (10)

    static VALUE
    enumerator_feed(VALUE obj, VALUE v)
    {
        struct enumerator *e = enumerator_ptr(obj);

        if (!UNDEF_P(e->feedvalue)) {
            rb_raise(rb_eTypeError, "feed value already set");
        }
        e->feedvalue = v;

        return Qnil;
    }

inspect → string click to toggle source

Creates a printable version of e.

    static VALUE
    enumerator_inspect(VALUE obj)
    {
        return rb_exec_recursive(inspect_enumerator, obj, 0);
    }

next → object click to toggle source

Returns the next object in the enumerator, and move the internal position forward. When the position reached at the end, `StopIteration` is raised.

### Example¶ ↑

    a = [1,2,3]
    e = a.to_enum
    p e.next   #=> 1
    p e.next   #=> 2
    p e.next   #=> 3
    p e.next   #raises StopIteration

See class-level notes about external iterators.

    static VALUE
    enumerator_next(VALUE obj)
    {
        VALUE vs = enumerator_next_values(obj);
        return ary2sv(vs, 0);
    }

next_values → array click to toggle source

Returns the next object as an array in the enumerator, and move the internal position forward. When the position reached at the end, `StopIteration` is raised.

See class-level notes about external iterators.

This method can be used to distinguish `yield` and `yield nil`.

### Example¶ ↑

    o = Object.new
    def o.each
      yield
      yield 1
      yield 1, 2
      yield nil
      yield [1, 2]
    end
    e = o.to_enum
    p e.next_values
    p e.next_values
    p e.next_values
    p e.next_values
    p e.next_values
    e = o.to_enum
    p e.next
    p e.next
    p e.next
    p e.next
    p e.next

    ## yield args       next_values      next
    #  yield            []               nil
    #  yield 1          [1]              1
    #  yield 1, 2       [1, 2]           [1, 2]
    #  yield nil        [nil]            nil
    #  yield [1, 2]     [[1, 2]]         [1, 2]

    static VALUE
    enumerator_next_values(VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);
        VALUE vs;

        if (!UNDEF_P(e->lookahead)) {
            vs = e->lookahead;
            e->lookahead = Qundef;
            return vs;
        }

        return get_next_values(obj, e);
    }

peek → object click to toggle source

Returns the next object in the enumerator, but doesn’t move the internal position forward. If the position is already at the end, `StopIteration` is raised.

See class-level notes about external iterators.

### Example¶ ↑

    a = [1,2,3]
    e = a.to_enum
    p e.next   #=> 1
    p e.peek   #=> 2
    p e.peek   #=> 2
    p e.peek   #=> 2
    p e.next   #=> 2
    p e.next   #=> 3
    p e.peek   #raises StopIteration

    static VALUE
    enumerator_peek(VALUE obj)
    {
        VALUE vs = enumerator_peek_values(obj);
        return ary2sv(vs, 1);
    }

peek_values → array click to toggle source

Returns the next object as an array, similar to `Enumerator#next_values`, but doesn’t move the internal position forward. If the position is already at the end, `StopIteration` is raised.

See class-level notes about external iterators.

### Example¶ ↑

    o = Object.new
    def o.each
      yield
      yield 1
      yield 1, 2
    end
    e = o.to_enum
    p e.peek_values    #=> []
    e.next
    p e.peek_values    #=> [1]
    p e.peek_values    #=> [1]
    e.next
    p e.peek_values    #=> [1, 2]
    e.next
    p e.peek_values    # raises StopIteration

    static VALUE
    enumerator_peek_values_m(VALUE obj)
    {
        return rb_ary_dup(enumerator_peek_values(obj));
    }

rewind → e click to toggle source

Rewinds the enumeration sequence to the beginning.

If the enclosed object responds to a “rewind” method, it is called.

    static VALUE
    enumerator_rewind(VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);

        rb_check_funcall(e->obj, id_rewind, 0, 0);

        e->fib = 0;
        e->dst = Qnil;
        e->lookahead = Qundef;
        e->feedvalue = Qundef;
        e->stop_exc = Qfalse;
        return obj;
    }

size → int, Float::INFINITY or nil click to toggle source

Returns the size of the enumerator, or `nil` if it can’t be calculated lazily.

    (1..100).to_a.permutation(4).size # => 94109400
    loop.size # => Float::INFINITY
    (1..100).drop_while.size # => nil

    static VALUE
    enumerator_size(VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);
        int argc = 0;
        const VALUE *argv = NULL;
        VALUE size;

        if (e->procs) {
            struct generator *g = generator_ptr(e->obj);
            VALUE receiver = rb_check_funcall(g->obj, id_size, 0, 0);
            long i = 0;

            for (i = 0; i < RARRAY_LEN(e->procs); i++) {
                VALUE proc = RARRAY_AREF(e->procs, i);
                struct proc_entry *entry = proc_entry_ptr(proc);
                lazyenum_size_func *size_fn = entry->fn->size;
                if (!size_fn) {
                    return Qnil;
                }
                receiver = (*size_fn)(proc, receiver);
            }
            return receiver;
        }

        if (e->size_fn) {
            return (*e->size_fn)(e->obj, e->args, obj);
        }
        if (e->args) {
            argc = (int)RARRAY_LEN(e->args);
            argv = RARRAY_CONST_PTR(e->args);
        }
        size = rb_check_funcall_kw(e->size, id_call, argc, argv, e->kw_splat);
        if (!UNDEF_P(size)) return size;
        return e->size;
    }

with_index(offset = 0) {|(*args), idx| ... } click to toggle source
with_index(offset = 0)

Iterates the given block for each element with an index, which starts from `offset`. If no block is given, returns a new `Enumerator` that includes the index, starting from `offset`

`offset`

  the starting index to use

    static VALUE
    enumerator_with_index(int argc, VALUE *argv, VALUE obj)
    {
        VALUE memo;

        rb_check_arity(argc, 0, 1);
        RETURN_SIZED_ENUMERATOR(obj, argc, argv, enumerator_enum_size);
        memo = (!argc || NIL_P(memo = argv[0])) ? INT2FIX(0) : rb_to_int(memo);
        return enumerator_block_call(obj, enumerator_with_index_i, (VALUE)MEMO_NEW(memo, 0, 0));
    }

with_object(obj) {|(*args), obj| ... }
with_object(obj)

Iterates the given block for each element with an arbitrary object, `obj`, and returns `obj`

If no block is given, returns a new `Enumerator`.

### Example¶ ↑

    to_three = Enumerator.new do |y|
      3.times do |x|
        y << x
      end
    end

    to_three_with_string = to_three.with_object("foo")
    to_three_with_string.each do |x,string|
      puts "#{string}: #{x}"
    end

    # => foo: 0
    # => foo: 1
    # => foo: 2

Alias for: each_with_object