class Enumerator - Documentation for Ruby 4.1
Description: class Enumerator: Enumerator@External+Iteration]. Enumerator@Internal+Iteration].
Keywords: ruby,class,Enumerator

🔍

No recent searches

# class Enumerator

Class Enumerator supports:

- External iteration.

- Internal iteration.

An Enumerator may be created by the following methods:

- `Object#to_enum`.

- `Object#enum_for`.

- `Enumerator.new`.

In addition, certain Ruby methods return Enumerator objects: a Ruby iterator method that accepts a block may return an Enumerator if no block is given. There are many such methods, for example, in classes `Array` and `Hash`. (In the documentation for those classes, search for `new_enumerator`.)

## Internal Iteration

In _internal iteration_, an iterator method drives the iteration and the caller’s block handles the processing; this example uses method `each_with_index`:

    words = %w[foo bar baz] # => ["foo", "bar", "baz"]
    enumerator = words.each # => #<Enumerator: ...>
    enumerator.each_with_index {|word, i| puts "#{i}: #{word}" }
    0: foo
    1: bar
    2: baz

Iterator methods in class Enumerator include:

- `each`: passes each item to the block.

- `each_with_index`: passes each item and its index to the block.

- `each_with_object` (aliased as `with_object`): passes each item and a given object to the block.

- `with_index`: like `each_with_index`, but starting at a given offset (instead of zero).

Class Enumerator includes module `Enumerable`, which provides many more iterator methods.

## External Iteration

In _external iteration_, the user’s program both drives the iteration and handles the processing in stream-like fashion; this example uses method `next`:

    words = %w[foo bar baz]
    enumerator = words.each
    enumerator.next # => "foo"
    enumerator.next # => "bar"
    enumerator.next # => "baz"
    enumerator.next # Raises StopIteration: iteration reached an end

External iteration methods in class Enumerator include:

- `feed`: sets the value that is next to be returned.

- `next`: returns the next value and increments the position.

- `next_values`: returns the next value in a 1-element array and increments the position.

- `peek`: returns the next value but does not increment the position.

- `peek_values`: returns the next value in a 1-element array but does not increment the position.

- `rewind`: sets the position to zero.

Each of these methods raises `FrozenError` if called from a frozen Enumerator.

## External Iteration and Fiber

External iteration that uses `Fiber` differs significantly from internal iteration:

- Using Fiber adds some overhead compared to internal enumeration.

- The stacktrace will only include the stack from the Enumerator, not above.

- Fiber-local variables are not inherited inside the Enumerator Fiber, which instead starts with no Fiber-local variables.

- Fiber storage variables are inherited and are designed to handle Enumerator Fibers. Assigning to a Fiber storage variable only affects the current Fiber, so if you want to change state in the caller Fiber of the Enumerator Fiber, you need to use an extra indirection (e.g., use some object in the Fiber storage variable and mutate some ivar of it).

Concretely:

    Thread.current[:fiber_local] = 1
    Fiber[:storage_var] = 1
    e = Enumerator.new do |y|
      p Thread.current[:fiber_local] # for external iteration: nil, for internal iteration: 1
      p Fiber[:storage_var] # => 1, inherited
      Fiber[:storage_var] += 1
      y << 42
    end

    p e.next # => 42
    p Fiber[:storage_var] # => 1 (it ran in a different Fiber)

    e.each { p _1 }
    p Fiber[:storage_var] # => 2 (it ran in the same Fiber/"stack" as the current Fiber)

## Converting External Iteration to Internal Iteration

You can use an external iterator to implement an internal iterator as follows:

    def ext_each(e)
      while true
        begin
          vs = e.next_values
        rescue StopIteration
          return $!.result
        end
        y = yield(*vs)
        e.feed y
      end
    end

    o = Object.new

    def o.each
      puts yield
      puts yield(1)
      puts yield(1, 2)
      3
    end

    # use o.each as an internal iterator directly.
    puts o.each {|*x| puts x; [:b, *x] }
    # => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3

    # convert o.each to an external iterator for
    # implementing an internal iterator.
    puts ext_each(o.to_enum) {|*x| puts x; [:b, *x] }
    # => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3

### Public Class Methods

new(size = nil) {|yielder| ... }
Source

    static VALUE
    enumerator_initialize(int argc, VALUE *argv, VALUE obj)
    {
        VALUE iter = rb_block_proc();
        VALUE recv = generator_init(generator_allocate(rb_cGenerator), iter);
        VALUE arg0 = rb_check_arity(argc, 0, 1) ? argv[0] : Qnil;
        VALUE size = convert_to_feasible_size_value(arg0);

        return enumerator_init(obj, recv, sym_each, 0, 0, 0, size, false);
    }

Returns a new Enumerator object that can be used for iteration.

The given block defines the iteration; it is called with a “yielder” object that can yield an object via a call to method `yielder.yield`:

    fib = Enumerator.new do |yielder|
      n = next_n = 1
      while true do
        yielder.yield(n)
        n, next_n = next_n, n + next_n
      end
    end

    fib.take(10) # => [1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

Parameter `size` specifies how the size is to be calculated (see `size`); it can either be a value or a callable object:

    Enumerator.new{}.size          # => nil
    Enumerator.new(42){}.size      # => 42
    Enumerator.new(-> {42}){}.size # => 42

produce(initial = nil, size: nil) { |prev| block } → enumerator
Source

    static VALUE
    enumerator_s_produce(int argc, VALUE *argv, VALUE klass)
    {
        VALUE init, producer, opts, size;
        ID keyword_ids[1];

        if (!rb_block_given_p()) rb_raise(rb_eArgError, "no block given");

        keyword_ids[0] = rb_intern("size");
        rb_scan_args_kw(RB_SCAN_ARGS_LAST_HASH_KEYWORDS, argc, argv, "01:", &init, &opts);
        rb_get_kwargs(opts, keyword_ids, 0, 1, &size);

        size = UNDEF_P(size) ? DBL2NUM(HUGE_VAL) : convert_to_feasible_size_value(size);

        if (argc == 0 || (argc == 1 && !NIL_P(opts))) {
            init = Qundef;
        }

        producer = producer_init(producer_allocate(rb_cEnumProducer), init, rb_block_proc(), size);

        return rb_enumeratorize_with_size_kw(producer, sym_each, 0, 0, producer_size, RB_NO_KEYWORDS);
    }

Creates an infinite enumerator from any block, just called over and over. The result of the previous iteration is passed to the next one. If `initial` is provided, it is passed to the first iteration, and becomes the first element of the enumerator; if it is not provided, the first iteration receives `nil`, and its result becomes the first element of the iterator.

Raising `StopIteration` from the block stops an iteration.

    Enumerator.produce(1, &:succ)   # => enumerator of 1, 2, 3, 4, ....

    Enumerator.produce { rand(10) } # => infinite random number sequence

    ancestors = Enumerator.produce(node) { |prev| node = prev.parent or raise StopIteration }
    enclosing_section = ancestors.find { |n| n.type == :section }

Using `::produce` together with `Enumerable` methods like `Enumerable#detect`, `Enumerable#slice_after`, `Enumerable#take_while` can provide Enumerator-based alternatives for `while` and `until` cycles:

    # Find next Tuesday
    require "date"
    Enumerator.produce(Date.today, &:succ).detect(&:tuesday?)

    # Simple lexer:
    require "strscan"
    scanner = StringScanner.new("7+38/6")
    PATTERN = %r{\d+|[-/+*]}
    Enumerator.produce { scanner.scan(PATTERN) }.slice_after { scanner.eos? }.first
    # => ["7", "+", "38", "/", "6"]

The optional `size` keyword argument specifies the size of the enumerator, which can be retrieved by `Enumerator#size`. It can be an integer, `Float::INFINITY`, a callable object (such as a lambda), or `nil` to indicate unknown size. When not specified, the size defaults to `Float::INFINITY`.

    # Infinite enumerator
    enum = Enumerator.produce(1, size: Float::INFINITY, &:succ)
    enum.size  # => Float::INFINITY

    # Finite enumerator with known/computable size
    abs_dir = File.expand_path("./baz") # => "/foo/bar/baz"
    traverser = Enumerator.produce(abs_dir, size: -> { abs_dir.count("/") + 1 }) {
      raise StopIteration if it == "/"
      File.dirname(it)
    }
    traverser.size  # => 4

    # Finite enumerator with unknown size
    calendar = Enumerator.produce(Date.today, size: nil) {
      it.monday? ? raise(StopIteration) : it + 1
    }
    calendar.size  # => nil

product(*enums) → enumerator
product(*enums) { |elts| ... } → enumerator
Source

    static VALUE
    enumerator_s_product(int argc, VALUE *argv, VALUE klass)
    {
        VALUE enums = Qnil, options = Qnil, block = Qnil;

        rb_scan_args(argc, argv, "*:&", &enums, &options, &block);

        if (!NIL_P(options) && !RHASH_EMPTY_P(options)) {
            rb_exc_raise(rb_keyword_error_new("unknown", rb_hash_keys(options)));
        }

        VALUE obj = enum_product_initialize(argc, argv, enum_product_allocate(rb_cEnumProduct));

        if (!NIL_P(block)) {
            enum_product_run(obj, block);
            return Qnil;
        }

        return obj;
    }

Generates a new enumerator object that generates a Cartesian product of given enumerable objects. This is equivalent to `Enumerator::Product.new`.

    e = Enumerator.product(1..3, [4, 5])
    e.to_a #=> [[1, 4], [1, 5], [2, 4], [2, 5], [3, 4], [3, 5]]
    e.size #=> 6

When a block is given, calls the block with each N-element array generated and returns `nil`.

### Public Instance Methods

e + enum → enumerator
Source

    static VALUE
    enumerator_plus(VALUE obj, VALUE eobj)
    {
        return new_enum_chain(rb_ary_new_from_args(2, obj, eobj));
    }

Returns an enumerator object generated from this enumerator and a given enumerable.

    e = (1..3).each + [4, 5]
    e.to_a #=> [1, 2, 3, 4, 5]

each { |elm| block } → obj
each → enum
each(*appending_args) { |elm| block } → obj
each(*appending_args) → an_enumerator
Source

    static VALUE
    enumerator_each(int argc, VALUE *argv, VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);

        if (argc > 0) {
            VALUE args = (e = enumerator_ptr(obj = rb_obj_dup(obj)))->args;
            if (args) {
    #if SIZEOF_INT < SIZEOF_LONG
                /* check int range overflow */
                rb_long2int(RARRAY_LEN(args) + argc);
    #endif
                args = rb_ary_dup(args);
                rb_ary_cat(args, argv, argc);
            }
            else {
                args = rb_ary_new4(argc, argv);
            }
            RB_OBJ_WRITE(obj, &e->args, args);
            e->size = Qnil;
            e->size_fn = 0;
        }
        if (!rb_block_given_p()) return obj;

        if (!lazy_precheck(e->procs)) return Qnil;

        return enumerator_block_call(obj, 0, obj);
    }

Iterates over the block according to how this `Enumerator` was constructed. If no block and no arguments are given, returns self.

### Examples

    "Hello, world!".scan(/\w+/)                     #=> ["Hello", "world"]
    "Hello, world!".to_enum(:scan, /\w+/).to_a      #=> ["Hello", "world"]
    "Hello, world!".to_enum(:scan).each(/\w+/).to_a #=> ["Hello", "world"]

    obj = Object.new

    def obj.each_arg(a, b=:b, *rest)
      yield a
      yield b
      yield rest
      :method_returned
    end

    enum = obj.to_enum :each_arg, :a, :x

    enum.each.to_a                  #=> [:a, :x, []]
    enum.each.equal?(enum)          #=> true
    enum.each { |elm| elm }         #=> :method_returned

    enum.each(:y, :z).to_a          #=> [:a, :x, [:y, :z]]
    enum.each(:y, :z).equal?(enum)  #=> false
    enum.each(:y, :z) { |elm| elm } #=> :method_returned

each_with_index {|(*args), idx| ... }
each_with_index
Source

    static VALUE
    enumerator_each_with_index(VALUE obj)
    {
        return enumerator_with_index(0, NULL, obj);
    }

Same as `Enumerator#with_index(0)`, i.e. there is no starting offset.

If no block is given, a new `Enumerator` is returned that includes the index.

each_with_object(obj) {|(*args), obj| ... }
each_with_object(obj)
Source

    static VALUE
    enumerator_with_object(VALUE obj, VALUE memo)
    {
        RETURN_SIZED_ENUMERATOR(obj, 1, &memo, enumerator_enum_size);
        enumerator_block_call(obj, enumerator_with_object_i, memo);

        return memo;
    }

Iterates the given block for each element with an arbitrary object, `obj`, and returns `obj`

If no block is given, returns a new `Enumerator`.

### Example

    to_three = Enumerator.new do |y|
      3.times do |x|
        y << x
      end
    end

    to_three_with_string = to_three.with_object("foo")
    to_three_with_string.each do |x,string|
      puts "#{string}: #{x}"
    end

    # => foo: 0
    # => foo: 1
    # => foo: 2

Also aliased as: with_object
feed obj → nil
Source

    static VALUE
    enumerator_feed(VALUE obj, VALUE v)
    {
        struct enumerator *e = enumerator_ptr(obj);

        rb_check_frozen(obj);

        if (!UNDEF_P(e->feedvalue)) {
            rb_raise(rb_eTypeError, "feed value already set");
        }
        RB_OBJ_WRITE(obj, &e->feedvalue, v);

        return Qnil;
    }

Sets the value to be returned by the next yield inside `e`.

If the value is not set, the yield returns nil.

This value is cleared after being yielded.

    # Array#map passes the array's elements to "yield" and collects the
    # results of "yield" as an array.
    # Following example shows that "next" returns the passed elements and
    # values passed to "feed" are collected as an array which can be
    # obtained by StopIteration#result.
    e = [1,2,3].map
    p e.next           #=> 1
    e.feed "a"
    p e.next           #=> 2
    e.feed "b"
    p e.next           #=> 3
    e.feed "c"
    begin
      e.next
    rescue StopIteration
      p $!.result      #=> ["a", "b", "c"]
    end

    o = Object.new
    def o.each
      x = yield         # (2) blocks
      p x               # (5) => "foo"
      x = yield         # (6) blocks
      p x               # (8) => nil
      x = yield         # (9) blocks
      p x               # not reached w/o another e.next
    end

    e = o.to_enum
    e.next              # (1)
    e.feed "foo"        # (3)
    e.next              # (4)
    e.next              # (7)
                        # (10)

inspect → string
Source

    static VALUE
    enumerator_inspect(VALUE obj)
    {
        return rb_exec_recursive(inspect_enumerator, obj, 0);
    }

Creates a printable version of e.

next → object
Source

    static VALUE
    enumerator_next(VALUE obj)
    {
        VALUE vs = enumerator_next_values(obj);
        return ary2sv(vs, 0);
    }

Returns the next object in the enumerator, and move the internal position forward. When the position reached at the end, `StopIteration` is raised.

### Example

    a = [1,2,3]
    e = a.to_enum
    p e.next   #=> 1
    p e.next   #=> 2
    p e.next   #=> 3
    p e.next   #raises StopIteration

See class-level notes about external iterators.

next_values → array
Source

    static VALUE
    enumerator_next_values(VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);
        VALUE vs;

        rb_check_frozen(obj);

        if (!UNDEF_P(e->lookahead)) {
            vs = e->lookahead;
            e->lookahead = Qundef;
            return vs;
        }

        return get_next_values(obj, e);
    }

Returns the next object as an array in the enumerator, and move the internal position forward. When the position reached at the end, `StopIteration` is raised.

See class-level notes about external iterators.

This method can be used to distinguish `yield` and `yield nil`.

### Example

    o = Object.new
    def o.each
      yield
      yield 1
      yield 1, 2
      yield nil
      yield [1, 2]
    end
    e = o.to_enum
    p e.next_values
    p e.next_values
    p e.next_values
    p e.next_values
    p e.next_values
    e = o.to_enum
    p e.next
    p e.next
    p e.next
    p e.next
    p e.next

    ## yield args       next_values      next
    #  yield            []               nil
    #  yield 1          [1]              1
    #  yield 1, 2       [1, 2]           [1, 2]
    #  yield nil        [nil]            nil
    #  yield [1, 2]     [[1, 2]]         [1, 2]

peek → object
Source

    static VALUE
    enumerator_peek(VALUE obj)
    {
        VALUE vs = enumerator_peek_values(obj);
        return ary2sv(vs, 1);
    }

Returns the next object in the enumerator, but doesn’t move the internal position forward. If the position is already at the end, `StopIteration` is raised.

See class-level notes about external iterators.

### Example

    a = [1,2,3]
    e = a.to_enum
    p e.next   #=> 1
    p e.peek   #=> 2
    p e.peek   #=> 2
    p e.peek   #=> 2
    p e.next   #=> 2
    p e.next   #=> 3
    p e.peek   #raises StopIteration

peek_values → array
Source

    static VALUE
    enumerator_peek_values_m(VALUE obj)
    {
        return rb_ary_dup(enumerator_peek_values(obj));
    }

Returns the next object as an array, similar to `Enumerator#next_values`, but doesn’t move the internal position forward. If the position is already at the end, `StopIteration` is raised.

See class-level notes about external iterators.

### Example

    o = Object.new
    def o.each
      yield
      yield 1
      yield 1, 2
    end
    e = o.to_enum
    p e.peek_values    #=> []
    e.next
    p e.peek_values    #=> [1]
    p e.peek_values    #=> [1]
    e.next
    p e.peek_values    #=> [1, 2]
    e.next
    p e.peek_values    # raises StopIteration

rewind → e
Source

    static VALUE
    enumerator_rewind(VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);

        rb_check_frozen(obj);

        rb_check_funcall(e->obj, id_rewind, 0, 0);

        e->fib = 0;
        e->dst = Qnil;
        e->lookahead = Qundef;
        e->feedvalue = Qundef;
        e->stop_exc = Qfalse;
        return obj;
    }

Rewinds the enumeration sequence to the beginning.

If the enclosed object responds to a “rewind” method, it is called.

size → int, Float::INFINITY or nil
Source

    static VALUE
    enumerator_size(VALUE obj)
    {
        struct enumerator *e = enumerator_ptr(obj);
        int argc = 0;
        const VALUE *argv = NULL;
        VALUE size;

        if (e->procs) {
            struct generator *g = generator_ptr(e->obj);
            VALUE receiver = rb_check_funcall(g->obj, id_size, 0, 0);
            long i = 0;

            for (i = 0; i < RARRAY_LEN(e->procs); i++) {
                VALUE proc = RARRAY_AREF(e->procs, i);
                struct proc_entry *entry = proc_entry_ptr(proc);
                lazyenum_size_func *size_fn = entry->fn->size;
                if (!size_fn) {
                    return Qnil;
                }
                receiver = (*size_fn)(proc, receiver);
            }
            return receiver;
        }

        if (e->size_fn) {
            return (*e->size_fn)(e->obj, e->args, obj);
        }
        if (e->args) {
            argc = (int)RARRAY_LEN(e->args);
            argv = RARRAY_CONST_PTR(e->args);
        }
        size = rb_check_funcall_kw(e->size, id_call, argc, argv, e->kw_splat);
        if (!UNDEF_P(size)) return size;
        return e->size;
    }

Returns the size of the enumerator, or `nil` if it can’t be calculated lazily.

    (1..100).to_a.permutation(4).size # => 94109400
    loop.size # => Float::INFINITY
    (1..100).drop_while.size # => nil

Note that enumerator size might be inaccurate, and should be rather treated as a hint. For example, there is no check that the size provided to `::new` is accurate:

    e = Enumerator.new(5) { |y| 2.times { y << it} }
    e.size # => 5
    e.to_a.size # => 2

Another example is an enumerator created by `::produce` without a `size` argument. Such enumerators return `Infinity` for size, but this is inaccurate if the passed block raises StopIteration:

    e = Enumerator.produce(1) { it + 1 }
    e.size # => Infinity

    e = Enumerator.produce(1) { it > 3 ? raise(StopIteration) : it + 1 }
    e.size # => Infinity
    e.to_a.size # => 4

with_index(offset = 0) {|(*args), idx| ... }
with_index(offset = 0)
Source

    static VALUE
    enumerator_with_index(int argc, VALUE *argv, VALUE obj)
    {
        VALUE memo;

        rb_check_arity(argc, 0, 1);
        RETURN_SIZED_ENUMERATOR(obj, argc, argv, enumerator_enum_size);
        memo = (!argc || NIL_P(memo = argv[0])) ? INT2FIX(0) : rb_to_int(memo);
        return enumerator_block_call(obj, enumerator_with_index_i, (VALUE)rb_imemo_memo_new(memo, 0, 0));
    }

Iterates the given block for each element with an index, which starts from `offset`. If no block is given, returns a new `Enumerator` that includes the index, starting from `offset`

`offset`

  the starting index to use

with_object(obj) {|(*args), obj| ... }
with_object(obj)

Iterates the given block for each element with an arbitrary object, `obj`, and returns `obj`

If no block is given, returns a new `Enumerator`.

### Example

    to_three = Enumerator.new do |y|
      3.times do |x|
        y << x
      end
    end

    to_three_with_string = to_three.with_object("foo")
    to_three_with_string.each do |x,string|
      puts "#{string}: #{x}"
    end

    # => foo: 0
    # => foo: 1
    # => foo: 2

Alias for: each_with_object