Class: Fiber

Inherits:
Object show all
Defined in:
cont.c,
cont.c

Overview

Fibers are primitives for implementing light weight cooperative concurrency in Ruby. Basically they are a means of creating code blocks that can be paused and resumed, much like threads. The main difference is that they are never preempted and that the scheduling must be done by the programmer and not the VM.

As opposed to other stackless light weight concurrency models, each fiber comes with a stack. This enables the fiber to be paused from deeply nested function calls within the fiber block. See the ruby(1) manpage to configure the size of the fiber stack(s).

When a fiber is created it will not run automatically. Rather it must be explicitly asked to run using the Fiber#resume method. The code running inside the fiber can give up control by calling Fiber.yield in which case it yields control back to caller (the caller of the Fiber#resume).

Upon yielding or termination the Fiber returns the value of the last executed expression

For instance:

fiber = Fiber.new do
  Fiber.yield 1
  2
end

puts fiber.resume
puts fiber.resume
puts fiber.resume

produces

1
2
FiberError: dead fiber called

The Fiber#resume method accepts an arbitrary number of parameters, if it is the first call to #resume then they will be passed as block arguments. Otherwise they will be the return value of the call to Fiber.yield

Example:

fiber = Fiber.new do |first|
  second = Fiber.yield first + 2
end

puts fiber.resume 10
puts fiber.resume 1_000_000
puts fiber.resume "The fiber will be dead before I can cause trouble"

produces

12
1000000
FiberError: dead fiber called

Non-blocking Fibers

The concept of non-blocking fiber was introduced in Ruby 3.0. A non-blocking fiber, when reaching a operation that would normally block the fiber (like sleep, or wait for another process or I/O) will yield control to other fibers and allow the scheduler to handle blocking and waking up (resuming) this fiber when it can proceed.

For a Fiber to behave as non-blocking, it need to be created in Fiber.new with blocking: false (which is the default), and Fiber.scheduler should be set with Fiber.set_scheduler. If Fiber.scheduler is not set in the current thread, blocking and non-blocking fibers’ behavior is identical.

Ruby doesn’t provide a scheduler class: it is expected to be implemented by the user and correspond to Fiber::Scheduler.

There is also Fiber.schedule method, which is expected to immediately perform the given block in a non-blocking manner. Its actual implementation is up to the scheduler.

Defined Under Namespace

Classes: Pool, Scheduler

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#new(blocking: false, storage: true) {|*args| ... } ⇒ Object

Creates new Fiber. Initially, the fiber is not running and can be resumed with #resume. Arguments to the first #resume call will be passed to the block:

f = Fiber.new do |initial|
   current = initial
   loop do
     puts "current: #{current.inspect}"
     current = Fiber.yield
   end
end
f.resume(100)     # prints: current: 100
f.resume(1, 2, 3) # prints: current: [1, 2, 3]
f.resume          # prints: current: nil
# ... and so on ...

If blocking: false is passed to Fiber.new, and current thread has a Fiber.scheduler defined, the Fiber becomes non-blocking (see “Non-blocking Fibers” section in class docs).

If the storage is unspecified, the default is to inherit a copy of the storage from the current fiber. This is the same as specifying storage: true.

Fiber[:x] = 1
Fiber.new do
  Fiber[:x] # => 1
  Fiber[:x] = 2
end.resume
Fiber[:x] # => 1

If the given storage is nil, this function will lazy initialize the internal storage, which starts as an empty hash.

Fiber[:x] = "Hello World"
Fiber.new(storage: nil) do
  Fiber[:x] # nil
end

Otherwise, the given storage is used as the new fiber’s storage, and it must be an instance of Hash.

Explicitly using storage: true is currently experimental and may change in the future.

Yields:

  • (*args) 


2293
2294
2295
2296
2297
 
# File 'cont.c', line 2293

static VALUE
rb_fiber_initialize(int argc, VALUE* argv, VALUE self)
{
    return rb_fiber_initialize_kw(argc, argv, self, rb_keyword_given_p());
}

Class Method Details

.[](key) ⇒ Object

Returns the value of the fiber storage variable identified by key.

The key must be a symbol, and the value is set by Fiber#[]= or Fiber#store.

See also Fiber::[]=.



2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
 
# File 'cont.c', line 2124

static VALUE
rb_fiber_storage_aref(VALUE class, VALUE key)
{
    key = rb_to_symbol(key);

    VALUE storage = fiber_storage_get(fiber_current(), FALSE);
    if (storage == Qnil) return Qnil;

    return rb_hash_aref(storage, key);
}

.[]=(key) ⇒ Object

Assign value to the fiber storage variable identified by key. The variable is created if it doesn’t exist.

key must be a Symbol, otherwise a TypeError is raised.

See also Fiber::[].



2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
 
# File 'cont.c', line 2145

static VALUE
rb_fiber_storage_aset(VALUE class, VALUE key, VALUE value)
{
    key = rb_to_symbol(key);

    VALUE storage = fiber_storage_get(fiber_current(), value != Qnil);
    if (storage == Qnil) return Qnil;

    if (value == Qnil) {
        return rb_hash_delete(storage, key);
    }
    else {
        return rb_hash_aset(storage, key, value);
    }
}

.blocking {|fiber| ... } ⇒ Object

Forces the fiber to be blocking for the duration of the block. Returns the result of the block.

See the “Non-blocking fibers” section in class docs for details.

Yields:

  • (fiber) 


2774
2775
2776
2777
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787
 
# File 'cont.c', line 2774

VALUE
rb_fiber_blocking(VALUE class)
{
    VALUE fiber_value = rb_fiber_current();
    rb_fiber_t *fiber = fiber_ptr(fiber_value);

    // If we are already blocking, this is essentially a no-op:
    if (fiber->blocking) {
        return rb_yield(fiber_value);
    }
    else {
        return rb_ensure(fiber_blocking_yield, fiber_value, fiber_blocking_ensure, fiber_value);
    }
}

.blocking? ⇒ false, 1

Returns false if the current fiber is non-blocking. Fiber is non-blocking if it was created via passing blocking: false to Fiber.new, or via Fiber.schedule.

If the current Fiber is blocking, the method returns 1. Future developments may allow for situations where larger integers could be returned.

Note that, even if the method returns false, Fiber behaves differently only if Fiber.scheduler is set in the current thread.

See the “Non-blocking fibers” section in class docs for details.

Returns:

  • (false, 1) 


2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
 
# File 'cont.c', line 2807

static VALUE
rb_fiber_s_blocking_p(VALUE klass)
{
    rb_thread_t *thread = GET_THREAD();
    unsigned blocking = thread->blocking;

    if (blocking == 0)
        return Qfalse;

    return INT2NUM(blocking);
}

.current ⇒ Object

Returns the current fiber. If you are not running in the context of a fiber this method will return the root fiber.



3246
3247
3248
3249
3250
 
# File 'cont.c', line 3246

static VALUE
rb_fiber_s_current(VALUE klass)
{
    return rb_fiber_current();
}

.current_scheduler ⇒ Object?

Returns the Fiber scheduler, that was last set for the current thread with Fiber.set_scheduler if and only if the current fiber is non-blocking.

Returns:



2399
2400
2401
2402
2403
 
# File 'cont.c', line 2399

static VALUE
rb_fiber_current_scheduler(VALUE klass)
{
    return rb_fiber_scheduler_current();
}

.schedule {|*args| ... } ⇒ Object

The method is expected to immediately run the provided block of code in a separate non-blocking fiber.

puts "Go to sleep!"

Fiber.set_scheduler(MyScheduler.new)

Fiber.schedule do
  puts "Going to sleep"
  sleep(1)
  puts "I slept well"
end

puts "Wakey-wakey, sleepyhead"

Assuming MyScheduler is properly implemented, this program will produce:

Go to sleep!
Going to sleep
Wakey-wakey, sleepyhead
...1 sec pause here...
I slept well

…e.g. on the first blocking operation inside the Fiber (sleep(1)), the control is yielded to the outside code (main fiber), and at the end of that execution, the scheduler takes care of properly resuming all the blocked fibers.

Note that the behavior described above is how the method is expected to behave, actual behavior is up to the current scheduler’s implementation of Fiber::Scheduler#fiber method. Ruby doesn’t enforce this method to behave in any particular way.

If the scheduler is not set, the method raises RuntimeError (No scheduler is available!).

Yields:

  • (*args) 


2369
2370
2371
2372
2373
 
# File 'cont.c', line 2369

static VALUE
rb_fiber_s_schedule(int argc, VALUE *argv, VALUE obj)
{
    return rb_fiber_s_schedule_kw(argc, argv, rb_keyword_given_p());
}

.scheduler ⇒ Object?

Returns the Fiber scheduler, that was last set for the current thread with Fiber.set_scheduler. Returns nil if no scheduler is set (which is the default), and non-blocking fibers’ behavior is the same as blocking. (see “Non-blocking fibers” section in class docs for details about the scheduler concept).

Returns:



2385
2386
2387
2388
2389
 
# File 'cont.c', line 2385

static VALUE
rb_fiber_s_scheduler(VALUE klass)
{
    return rb_fiber_scheduler_get();
}

.set_scheduler(scheduler) ⇒ Object

Sets the Fiber scheduler for the current thread. If the scheduler is set, non-blocking fibers (created by Fiber.new with blocking: false, or by Fiber.schedule) call that scheduler’s hook methods on potentially blocking operations, and the current thread will call scheduler’s close method on finalization (allowing the scheduler to properly manage all non-finished fibers).

scheduler can be an object of any class corresponding to Fiber::Scheduler. Its implementation is up to the user.

See also the “Non-blocking fibers” section in class docs.



2421
2422
2423
2424
2425
 
# File 'cont.c', line 2421

static VALUE
rb_fiber_set_scheduler(VALUE klass, VALUE scheduler)
{
    return rb_fiber_scheduler_set(scheduler);
}

.yield(args, ...) ⇒ Object

Yields control back to the context that resumed the fiber, passing along any arguments that were passed to it. The fiber will resume processing at this point when #resume is called next. Any arguments passed to the next #resume will be the value that this Fiber.yield expression evaluates to.

Returns:



3137
3138
3139
3140
3141
 
# File 'cont.c', line 3137

static VALUE
rb_fiber_s_yield(int argc, VALUE *argv, VALUE klass)
{
    return rb_fiber_yield_kw(argc, argv, rb_keyword_given_p());
}

Instance Method Details

#alive? ⇒ Boolean

Returns true if the fiber can still be resumed (or transferred to). After finishing execution of the fiber block this method will always return false.

Returns:

  • (Boolean) 


2915
2916
2917
2918
2919
 
# File 'cont.c', line 2915

VALUE
rb_fiber_alive_p(VALUE fiber_value)
{
    return RBOOL(!FIBER_TERMINATED_P(fiber_ptr(fiber_value)));
}

#backtrace ⇒ Array #backtrace(start) ⇒ Array #backtrace(start, count) ⇒ Array #backtrace(start..end) ⇒ Array

Returns the current execution stack of the fiber. start, count and end allow to select only parts of the backtrace.

def level3
  Fiber.yield
end

def level2
  level3
end

def level1
  level2
end

f = Fiber.new { level1 }

# It is empty before the fiber started
f.backtrace
#=> []

f.resume

f.backtrace
#=> ["test.rb:2:in `yield'", "test.rb:2:in `level3'", "test.rb:6:in `level2'", "test.rb:10:in `level1'", "test.rb:13:in `block in <main>'"]
p f.backtrace(1) # start from the item 1
#=> ["test.rb:2:in `level3'", "test.rb:6:in `level2'", "test.rb:10:in `level1'", "test.rb:13:in `block in <main>'"]
p f.backtrace(2, 2) # start from item 2, take 2
#=> ["test.rb:6:in `level2'", "test.rb:10:in `level1'"]
p f.backtrace(1..3) # take items from 1 to 3
#=> ["test.rb:2:in `level3'", "test.rb:6:in `level2'", "test.rb:10:in `level1'"]

f.resume

# It is nil after the fiber is finished
f.backtrace
#=> nil

Overloads:



2988
2989
2990
2991
2992
 
# File 'cont.c', line 2988

static VALUE
rb_fiber_backtrace(int argc, VALUE *argv, VALUE fiber)
{
    return rb_vm_backtrace(argc, argv, &fiber_ptr(fiber)->cont.saved_ec);
}

#backtrace_locations ⇒ Array #backtrace_locations(start) ⇒ Array #backtrace_locations(start, count) ⇒ Array #backtrace_locations(start..end) ⇒ Array

Like #backtrace, but returns each line of the execution stack as a Thread::Backtrace::Location. Accepts the same arguments as #backtrace.

f = Fiber.new { Fiber.yield }
f.resume
loc = f.backtrace_locations.first
loc.label  #=> "yield"
loc.path   #=> "test.rb"
loc.lineno #=> 1

Overloads:

  • #backtrace_locations ⇒ Array

    Returns:

  • #backtrace_locations(start) ⇒ Array

    Returns:

  • #backtrace_locations(start, count) ⇒ Array

    Returns:

  • #backtrace_locations(start..end) ⇒ Array

    Returns:



3013
3014
3015
3016
3017
 
# File 'cont.c', line 3013

static VALUE
rb_fiber_backtrace_locations(int argc, VALUE *argv, VALUE fiber)
{
    return rb_vm_backtrace_locations(argc, argv, &fiber_ptr(fiber)->cont.saved_ec);
}

#blocking? ⇒ Boolean

Returns true if fiber is blocking and false otherwise. Fiber is non-blocking if it was created via passing blocking: false to Fiber.new, or via Fiber.schedule.

Note that, even if the method returns false, the fiber behaves differently only if Fiber.scheduler is set in the current thread.

See the “Non-blocking fibers” section in class docs for details.

Returns:

  • (Boolean) 


2728
2729
2730
2731
2732
 
# File 'cont.c', line 2728

VALUE
rb_fiber_blocking_p(VALUE fiber)
{
    return RBOOL(fiber_ptr(fiber)->blocking);
}

#kill ⇒ nil

Terminates the fiber by raising an uncatchable exception. It only terminates the given fiber and no other fiber, returning nil to another fiber if that fiber was calling #resume or #transfer.

Fiber#kill only interrupts another fiber when it is in Fiber.yield. If called on the current fiber then it raises that exception at the Fiber#kill call site.

If the fiber has not been started, transition directly to the terminated state.

If the fiber is already terminated, does nothing.

Raises FiberError if called on a fiber belonging to another thread.

Returns:

  • (nil) 


3216
3217
3218
3219
3220
3221
3222
3223
3224
3225
3226
3227
3228
3229
3230
3231
3232
3233
3234
3235
3236
3237
 
# File 'cont.c', line 3216

static VALUE
rb_fiber_m_kill(VALUE self)
{
    rb_fiber_t *fiber = fiber_ptr(self);

    if (fiber->killed) return Qfalse;
    fiber->killed = 1;

    if (fiber->status == FIBER_CREATED) {
        fiber->status = FIBER_TERMINATED;
    }
    else if (fiber->status != FIBER_TERMINATED) {
        if (fiber_current() == fiber) {
            fiber_check_killed(fiber);
        }
        else {
            fiber_raise(fiber_ptr(self), Qnil);
        }
    }

    return self;
}

#raise ⇒ Object #raise(string) ⇒ Object #raise(exception[, string [, array]]) ⇒ Object

Raises an exception in the fiber at the point at which the last Fiber.yield was called. If the fiber has not been started or has already run to completion, raises FiberError. If the fiber is yielding, it is resumed. If it is transferring, it is transferred into. But if it is resuming, raises FiberError.

With no arguments, raises a RuntimeError. With a single String argument, raises a RuntimeError with the string as a message. Otherwise, the first parameter should be the name of an Exception class (or an object that returns an Exception object when sent an exception message). The optional second parameter sets the message associated with the exception, and the third parameter is an array of callback information. Exceptions are caught by the rescue clause of begin...end blocks.

Raises FiberError if called on a Fiber belonging to another Thread.

See Kernel#raise for more information.

Overloads:



3193
3194
3195
3196
3197
 
# File 'cont.c', line 3193

static VALUE
rb_fiber_m_raise(int argc, VALUE *argv, VALUE self)
{
    return rb_fiber_raise(self, argc, argv);
}

#resume(args, ...) ⇒ Object

Resumes the fiber from the point at which the last Fiber.yield was called, or starts running it if it is the first call to #resume. Arguments passed to resume will be the value of the Fiber.yield expression or will be passed as block parameters to the fiber’s block if this is the first #resume.

Alternatively, when resume is called it evaluates to the arguments passed to the next Fiber.yield statement inside the fiber’s block or to the block value if it runs to completion without any Fiber.yield

Returns:



2936
2937
2938
2939
2940
 
# File 'cont.c', line 2936

static VALUE
rb_fiber_m_resume(int argc, VALUE *argv, VALUE fiber)
{
    return rb_fiber_resume_kw(fiber, argc, argv, rb_keyword_given_p());
}

#storage ⇒ Object

Returns a copy of the storage hash for the fiber. The method can only be called on the Fiber.current.



2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
 
# File 'cont.c', line 2037

static VALUE
rb_fiber_storage_get(VALUE self)
{
    storage_access_must_be_from_same_fiber(self);

    VALUE storage = fiber_storage_get(fiber_ptr(self), FALSE);

    if (storage == Qnil) {
        return Qnil;
    }
    else {
        return rb_obj_dup(storage);
    }
}

#storage=(hash) ⇒ Object

Sets the storage hash for the fiber. This feature is experimental and may change in the future. The method can only be called on the Fiber.current.

You should be careful about using this method as you may inadvertently clear important fiber-storage state. You should mostly prefer to assign specific keys in the storage using Fiber::[]=.

You can also use Fiber.new(storage: nil) to create a fiber with an empty storage.

Example:

while request = request_queue.pop
  # Reset the per-request state:
  Fiber.current.storage = nil
  handle_request(request)
end



2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
 
# File 'cont.c', line 2099

static VALUE
rb_fiber_storage_set(VALUE self, VALUE value)
{
    if (rb_warning_category_enabled_p(RB_WARN_CATEGORY_EXPERIMENTAL)) {
        rb_category_warn(RB_WARN_CATEGORY_EXPERIMENTAL,
          "Fiber#storage= is experimental and may be removed in the future!");
    }

    storage_access_must_be_from_same_fiber(self);
    fiber_storage_validate(value);

    fiber_ptr(self)->cont.saved_ec.storage = rb_obj_dup(value);
    return value;
}

#to_s ⇒ Object Also known as: inspect 



3252
3253
3254
3255
3256
3257
3258
3259
3260
3261
3262
3263
3264
3265
3266
3267
3268
3269
3270
3271
3272
3273
3274
3275
 
# File 'cont.c', line 3252

static VALUE
fiber_to_s(VALUE fiber_value)
{
    const rb_fiber_t *fiber = fiber_ptr(fiber_value);
    const rb_proc_t *proc;
    char status_info[0x20];

    if (fiber->resuming_fiber) {
        snprintf(status_info, 0x20, " (%s by resuming)", fiber_status_name(fiber->status));
    }
    else {
        snprintf(status_info, 0x20, " (%s)", fiber_status_name(fiber->status));
    }

    if (!rb_obj_is_proc(fiber->first_proc)) {
        VALUE str = rb_any_to_s(fiber_value);
        strlcat(status_info, ">", sizeof(status_info));
        rb_str_set_len(str, RSTRING_LEN(str)-1);
        rb_str_cat_cstr(str, status_info);
        return str;
    }
    GetProcPtr(fiber->first_proc, proc);
    return rb_block_to_s(fiber_value, &proc->block, status_info);
}

#transfer(args, ...) ⇒ Object

Transfer control to another fiber, resuming it from where it last stopped or starting it if it was not resumed before. The calling fiber will be suspended much like in a call to Fiber.yield.

The fiber which receives the transfer call treats it much like a resume call. Arguments passed to transfer are treated like those passed to resume.

The two style of control passing to and from fiber (one is #resume and Fiber::yield, another is #transfer to and from fiber) can’t be freely mixed.

  • If the Fiber’s lifecycle had started with transfer, it will never be able to yield or be resumed control passing, only finish or transfer back. (It still can resume other fibers that are allowed to be resumed.)

  • If the Fiber’s lifecycle had started with resume, it can yield or transfer to another Fiber, but can receive control back only the way compatible with the way it was given away: if it had transferred, it only can be transferred back, and if it had yielded, it only can be resumed back. After that, it again can transfer or yield.

If those rules are broken FiberError is raised.

For an individual Fiber design, yield/resume is easier to use (the Fiber just gives away control, it doesn’t need to think about who the control is given to), while transfer is more flexible for complex cases, allowing to build arbitrary graphs of Fibers dependent on each other.

Example:

manager = nil # For local var to be visible inside worker block

# This fiber would be started with transfer
# It can't yield, and can't be resumed
worker = Fiber.new { |work|
  puts "Worker: starts"
  puts "Worker: Performed #{work.inspect}, transferring back"
  # Fiber.yield     # this would raise FiberError: attempt to yield on a not resumed fiber
  # manager.resume  # this would raise FiberError: attempt to resume a resumed fiber (double resume)
  manager.transfer(work.capitalize)
}

# This fiber would be started with resume
# It can yield or transfer, and can be transferred
# back or resumed
manager = Fiber.new {
  puts "Manager: starts"
  puts "Manager: transferring 'something' to worker"
  result = worker.transfer('something')
  puts "Manager: worker returned #{result.inspect}"
  # worker.resume    # this would raise FiberError: attempt to resume a transferring fiber
  Fiber.yield        # this is OK, the fiber transferred from and to, now it can yield
  puts "Manager: finished"
}

puts "Starting the manager"
manager.resume
puts "Resuming the manager"
# manager.transfer  # this would raise FiberError: attempt to transfer to a yielding fiber
manager.resume

produces

Starting the manager
Manager: starts
Manager: transferring 'something' to worker
Worker: starts
Worker: Performed "something", transferring back
Manager: worker returned "Something"
Resuming the manager
Manager: finished

Returns:



3101
3102
3103
3104
3105
 
# File 'cont.c', line 3101

static VALUE
rb_fiber_m_transfer(int argc, VALUE *argv, VALUE self)
{
    return rb_fiber_transfer_kw(self, argc, argv, rb_keyword_given_p());
}