What are generators in PHP, and how do they compare to arrays?

PHP: What are generators in PHP, and how do they compare to arrays?

Generators are like little functions that let you iterate over it (for example with a foreach) and it will 'yield' a value on every iteration.

If you do foreach(range(1,10000000) as $i) {...} then PHP will probably crash, as it has to first 'create' the array (with items from 1 to 10000000 in it). This isn't terribly memory efficient.

A better way to do it is using a generator. It will 'output' (yield) one value at a time, so uses much less memory.

Here is an example of a generator - take note of the yield line.

  1. function range_generator($from, $to) {
  2.     for ($i = $from; $i <= $to; $i++) {
  3.         yield $i; // here!
  4.     }
  5. }

The yield $i is sort of the same as return $i, however the code that called range_generator() (i.e. the foreach loop) can keep asking it to yield again and again and again...

This range_generator() function is often called xrange() in other languages that have this function built in, BTW

The function above can be used like this:

  1. foreach(range_generator(1,10000000) as $i) {
  2.         echo $i;
  3.  
  4.         }
  1. The foreach loop starts, and gives the generator two parameters, 1 and 10000000.
  2. The generator 'beings', and starts its own for loop. In its first iteration, it will yield 1.
  3. This is then passed back to the foreach ( as $i), and the code within the foreach loop is executed (echo $i).
  4. Once the first iteration of the foreach is complete, it will go back to the range generator, which will then finish its for loop (nothing else for it to do), and go into its own second iteration of that loop, until it reaches the yield $i again, where the pattern repeats.
  5. This carries on until the range_generator stops yielding a variables back to the foreach loop.

If you used the standard range(1,10000000) function then the array of 10000000 elements has to be created and stay in memory. Using this generator means that only one $i exists at a time, massively reducing memory load. If I use the standard range(1,10000000) function, I get a Allowed memory size of 134217728 bytes exhausted (tried to allocate 536870920 bytes) exception.

The memory usage with a generators is constant, no matter how many times it iterates. It isn't really even fair to compare an array and a generator, as they aren't really the same thing. But you can loop them in things like a foreach loop.

How to send data to the generator

When you work with generators, you are actually working with the Generator class. There are a number of methods available to you:

  • public mixed current() - Get the yielded value
  • public mixed getReturn() - Get the return value of a generator. This is used after you have finished using the generator - as soon as you return anything (null, a value) the generator will stop yielding
  • public mixed key() - Get the yielded key
  • public void next() - Resume execution of the generator (same as calling Generator::send() with NULL as argument)
  • public void rewind() - Rewind the iterator. N.B. If iteration has already begun, this will throw an exception.
  • public mixed send( $value ) - Sends the given value to the generator as the result of the current yield expression and resumes execution of the generator.(see below for more details)
  • public mixed throw( Throwable $exception ) -- Throw an exception into the generator, and then resumes execution of the generator. The behavior will be the same as if the current yield expression was replaced with a throw $exception statement
  • public bool valid() - Check if the iterator has been closed
  • public void __wakeup() - not really used - it will just throw an exception as you cannot serialise generators.

I think the most important one to know is the send($val) method. Here is an example:

  1. <?php
  2. function printer() {
  3.     echo "I'm printer!".PHP_EOL;
  4.     while (true) {
  5.         $string = yield;
  6.         echo $string.PHP_EOL;
  7.     }
  8. }
  9.  
  10. $printer = printer();
  11. $printer->send('Hello world!');
  12. $printer->send('Bye world!');
  13. ?>

This will output the following:

I'm printer!
Hello world!
Bye world!

When to use generators and their yielding feature

If you are ever doing something with a large amount of data and iterating over all of it, then your first thought should always be if you should be using a generator.

You don't want to be storing huge amounts of data in memory, especially if you are actually only working on one part (one line, one element, one document, one row) at a time.

Good times to use generators:

  • Dealing with things like database rows. It is fine to process a few hundred (even a few thousand) in a normal array based loop. But if you have a big data set then you will very quickly run out of memory. Put it in a generator and handle one row at a time.
  • When working with log files. Log files can easily be many GBs of text. Loading it all into memory at once, again, would be a bad idea. But cycling through it line by line means you won't face any memory issues.
webdevetc profile pic
webdevetc

I am a 29 year old backend web developer from London, mostly focusing on PHP and Laravel lately. This (webdevetc.com) is my blog where I write about some web development topics (PHP, Laravel, Javascript, and some server stuff). contact me here.



More...


Comments and discussion about What are generators in PHP, and how do they compare to arrays?

Found this interesting? Maybe you want to read some more in this series?

Or see other topics in the PHP language

Or see other languages/frameworks:
PHP Laravel Composer Apache CentOS and Linux Stuff WordPress General Webdev and Programming Stuff JavaScript
Or see random questions

How to run an artisan command from a controller

How to review blog posts (written by other users) before they get published on your WordPress blog?

Set PHP config Variables (like ini_set) with .htaccess

What is an example of an object literal?

What are macros in Laravel?

How to get the type of a variable in JS?

How to generate an array for a dropdown (with ['key' => 'value']) suitable for a dropdown, from a Laravel collection of Eloquent objects?

What are Apache MPMs?

How to replace whitespace with a single space

How to add the CSRF (Cross-site request forgery) token in Laravel?