3.6.0 release post #105
3.6.0 release post #105
Conversation
|
|
||
| A special thanks to Ewa Czechowska for getting this in to core. | ||
|
|
||
| ### Expectations: Scoped alised and negative matchers |
| --- | ||
|
|
||
| RSpec 3.6 has just been released! Given our commitment to | ||
| [semantic versioning](http://semver.org/), this should be a trivial |
There was a problem hiding this comment.
For readability I think this could be "this should be an easy upgrade"
There was a problem hiding this comment.
This is copied word for word from the previous post. I'll make your change, as I think it's better. We should standardise this blog post into a template.
| RSpec 3.6.0 comes with full support for Rails 5.1. There are no major changes to | ||
| the rails 5.1 API and so this upgrade should be fully smooth. Simply bumping to | ||
| the latest version of RSpec will bring the support to your app with no other | ||
| changes required on your part |
There was a problem hiding this comment.
Is it worth mentioning that system tests are not yet supported?
There was a problem hiding this comment.
This last sentence is missing a period.
There was a problem hiding this comment.
Is it worth mentioning that system tests are not yet supported?
Correct me if I'm wrong, but based on my reading of the system tests section of the 5.1 release announcement, it sounds like system tests are what rspec-rails users have had for years when they install capybara and write :feature specs.
Or is there more to system tests then that? If there's not, saying "system tests are not yet supported" sends the wrong message--it makes it sound like "system tests" are a compelling thing offered by the default rails testing setup that rspec-rails lacks. Instead, something like "Rails 5.1 added a feature called system tests, which brings parity to the feature tests rspec-rails and capybara have offered for years" might be better. (But only if that's accurate; I'm not sure if it is).
|
|
||
| ## Notable Changes | ||
|
|
||
| ### Core: Failure counts of errors occurring outside examples |
There was a problem hiding this comment.
IMO, the fact that we count failures is less interesting than the fact that we handle these errors at all. Before the changes in RSpec 3.6 (such as rspec/rspec-core#2323), errors that occurred in :suite hooks or while loading spec files would just crash the ruby interpreter and you'd get ruby's default error output including a full stacktrace. Now we handle these errors and provide nicely formatted output, including backtrace filtering, error-site code snippet, and mentioning the count of these errors in the summary output.
WDYT of this instead?
Core: Errors outside examples now handled and formatted well
In previous versions of RSpec, we allowed errors encountered while loading spec files or running :suite hooks to crash the ruby interpreter, giving you its default full-stacktrace output. In RSpec 3.6, we now handle all errors that occur outside examples, and format them nicely including a filtered backtrace and a code snippet for the site of the error. For example, an error in a before(:suite) hook is now formatted like this:
Thanks to Jon Rowe for assisting with this feature.
|
|
||
| As it currently stands RSpec will exit with code `0` indicating success if no | ||
| examples are defined. This configuration option makes it possible to cause RSpec | ||
| to exit with code `1` indicating failure, which is useful for CI environments. |
There was a problem hiding this comment.
I think it would be better to more explicitly explain why this is useful--i.e. it helps detect when you've misconfigured RSpec to look for specs in the wrong place or with the wrong pattern and it therefore can't find any. Instead of always passing, this makes it fail in that case.
| ~~~ | ||
|
|
||
| A special thanks to Ewa Czechowska for getting this in to core. | ||
|
|
There was a problem hiding this comment.
Before moving on to the expectations changes, there's one more notable core changes, IMO: color is automatically enabled if RSpec is outputting to a TTY, so there's no longer a need to set --color to opt-in to this behavior. If you really want color (even if the output isn't a TTY), you can pass --force-color, or --no-color to disable color output. Can you add a section on that?
| end | ||
| ~~~ | ||
|
|
||
| here, this test would fail if the `without_partial_double_verification` call was |
|
|
||
| here, this test would fail if the `without_partial_double_verification` call was | ||
| not made, because we are stubbing the `call` method on the `Widget` object | ||
| with an argument count that is different to the implementation. |
There was a problem hiding this comment.
The explanation for this feature makes it sound like this would never be useful. Can you provide a rationale for why this was added (I think it's useful in some kind of rspec-rails context?)
| RSpec 3.6.0 comes with full support for Rails 5.1. There are no major changes to | ||
| the rails 5.1 API and so this upgrade should be fully smooth. Simply bumping to | ||
| the latest version of RSpec will bring the support to your app with no other | ||
| changes required on your part |
There was a problem hiding this comment.
This last sentence is missing a period.
|
|
||
| ## Stats: | ||
|
|
||
| TODO when ready to release |
There was a problem hiding this comment.
To generate this, run rake "version_stats[v3.5.0...v3.6.0]" in rspec-dev once you've tagged all the repos with v3.6.0.
Also, we usually include a Docs section and a Release Notes section in these blog posts (take a look at the 3.5 or 3.4 posts for an example).
For the release notes, we have an rspec-dev task (release_notes) to assist. That said, it would be good to combine the release notes from all 3.6 betas here, which I don't think it does. Maybe you can combine those by hand as I did for the 3.5 blog post?
|
@myronmarston @JonRowe I've just pushed some points addressing the review feedback. Is anyone able to fill out the TODO in the rspec-expectations section? |
|
|
||
| RSpec continues to be a community-driven project with contributors | ||
| from all over the world. This release includes over XXX commits and YYY | ||
| merged pull requests from over 50 different contributors! |
There was a problem hiding this comment.
It's funny that you have the contributor count filled in but not the commits and PRs! Usually I fill this in after generating the stats as it is based on that.
| `before(:suite)` hook is now formatted like this: | ||
|
|
||
| <img width="479" alt="screen shot 2017-04-27 at 9 59 12 pm" | ||
| src="https://cloud.githubusercontent.com/assets/49391/25514870/cb9db6c8-2b94-11e7-952f-f26fd783512b.png"> |
There was a problem hiding this comment.
Would it be better to add the image to this repository instead of hotlinking to github's CDN? Ultimately, the site is hosted on github, of course, but there's something nice about keeping the image under source control, too...
| As it currently stands RSpec will exit with code `0` indicating success if no | ||
| examples are defined. This configuration option makes it possible to cause RSpec | ||
| to exit with code `1` indicating failure. This is useful in CI environments, it | ||
| helps detect when you've misconfigured RSpec to look for specs in the wrong |
There was a problem hiding this comment.
"it helps detect" reads like it's the start of a new sentence. If you want to keep it as one sentence, I think it helps to add a connecting word like "as":
This is useful in CI environments, as it helps detect...
| examples are defined. This configuration option makes it possible to cause RSpec | ||
| to exit with code `1` indicating failure. This is useful in CI environments, it | ||
| helps detect when you've misconfigured RSpec to look for specs in the wrong | ||
| place or with the wrong pattern. When this occurs we consider finding no specs |
There was a problem hiding this comment.
When this occurs
What is the "this" here? Do you mean when RSpec is misconfigured to look for example in the wrong place? Or when the user has configured it with config.fail_if_no_examples = true? Both are needed for the rest of the statement to be true but this sentence doesn't really explain that.
|
|
||
| For existing RSpec projects, that were initialized with either `bundle exec | ||
| rspec --init` or `bundle exec rails g rspec:install` you can find this setting | ||
| in your `.rspec` file if you'd like to change it. |
There was a problem hiding this comment.
I don't think we need this paragraph. The existing setting (--color) is a no-op in 3.6 as the behavior it enables is the default now. So the setting is effectively ignored, and the user can set --force-color or --no-color in whatever fashion they want, without regard for where --color may have been set previously.
|
|
||
| ### Expectations: Scoped aliased and negative matchers | ||
|
|
||
| TODO: is anyone more familiar with this feature who could full this out? |
There was a problem hiding this comment.
Basically, the idea is that you can now call alias_matcher or define_negated_matcher in an example group to limit the scope of the newly defined matchers to just that group, rather than them being defined globally for use from all groups.
|
|
||
| RSpec.describe Widget do | ||
| it 'can be stub with a mismatched arg count' do | ||
| without_partial_double_verification { |
There was a problem hiding this comment.
Why curlies here instead of do/end?
| situations where methods aren't already defined on the object, like in Rails, | ||
| when methods on `ActiveRecord` objects are defined after database interactions. | ||
| With this feature, we can test interfaces to those objects while retaining | ||
| database isolation. |
There was a problem hiding this comment.
Is this needed for AR objects? I didn't think it helped then. (And in fact, I thought @JonRowe implemented some stuff for ActiveRecord models to make their dynamic nature cause no problems for verified doubles). Isn't the situation where this is needed helper specs or view specs or something? @JonRowe can you help here so we can accurately explain when this API is useful?
|
@myronmarston could you give this another look? |
| @@ -0,0 +1,144 @@ | |||
| --- | |||
There was a problem hiding this comment.
Don't forget to update the date in the filename!
|
|
||
| As it currently stands RSpec will exit with code `0` indicating success if no | ||
| examples are defined. This configuration option makes it possible to cause RSpec | ||
| to exit with code `1` indicating failure. This is useful in CI environments, as |
There was a problem hiding this comment.
This configuration option makes it possible to cause RSpec to exit...
IMO, this is worded a bit awkwardly (particularly "makes it possible to cause"). I think this reads better:
This option allows you to configure RSpec to exit...
| to exit with code `1` indicating failure. This is useful in CI environments, as | ||
| it helps detect when you've misconfigured RSpec to look for specs in the wrong | ||
| place or with the wrong pattern. When no specs are found and | ||
| `config.fail_if_no_examples is set` we consider this to be an error as opposed |
There was a problem hiding this comment.
The 2nd back tick should end at examples, not is set.
|
|
||
| A special thanks to Ewa Czechowska for getting this in to core. | ||
|
|
||
| ### Core: Output coloring is set automatically if RSpec is outputting to a tty. |
There was a problem hiding this comment.
Thanks for spelling "color" the American way :).
| In RSpec 3, we added `alias_matcher`, allowing users to | ||
| [alias matchers](http://rspec.info/blog/2014/01/new-in-rspec-3-composable-matchers/#matcher-aliases) | ||
| for better readability. In 3.1 we added the ability to define | ||
| [http://rspec.info/blog/2014/09/rspec-3-1-has-been-released/#expectations-new-definenegatedmatcher-api](negated matchers) |
There was a problem hiding this comment.
You've switched the anchor text and link URL so it's not rendering correctly.
| matchers were always defined at the global scope. With this feature you are able | ||
| to invoke either `alias_matcher` or `define_negated_matcher` in the scope of an | ||
| example group (`describe`, `context`, etc). When doing so the new matcher that | ||
| you define will **only** be available for the duration of that example group: |
There was a problem hiding this comment.
"Duration" is kinda a confusing way to define scope, because it makes it sound like the scope is clock-based. Instead, how about:
When doing so the newly defined matcher will only be available to examples in that example group and any nested groups.
| end | ||
| end | ||
| end | ||
| ~~~ |
There was a problem hiding this comment.
Didn't a user contribute this feature? Can you add a thanks for that?
There was a problem hiding this comment.
They did, rspec/rspec-expectations#974, Markus Reiter.
|
|
||
| Here, this test would fail if the `without_partial_double_verification` call was | ||
| not made, because we are stubbing the `call` method on the `Widget` object with | ||
| an argument count that is different to the implementation. We added this feature |
There was a problem hiding this comment.
s/different to the implementation/different from the implementation/
(At least, the "to" sounds wrong to my ears, but maybe in British English that's correct...I'm not sure).
| changes required on your part. | ||
|
|
||
| Rails [system tests](http://weblog.rubyonrails.org/2017/4/27/Rails-5-1-final/) are not yet supported, | ||
| but we plan to add support for them in the near future. |
There was a problem hiding this comment.
Are system tests any different from feature specs w/ capybara?
There was a problem hiding this comment.
Only in that they're "blessed" and use a specific non-forked process server (run with puma) that allows for transactional testing.
| `before(:suite)` hook is now formatted like this: | ||
|
|
||
| <img width="479" alt="Errors outside example execution" | ||
| src="/images/blog/errors_outside_example.png"> |
There was a problem hiding this comment.
I think this image looks better w/o the width attribute. With it, it squishes the image in a way that doesn't look good to my eyes.
| not made, because we are stubbing the `call` method on the `Widget` object with | ||
| an argument count that is different to the implementation. We added this feature | ||
| specifically to address a problem with partial double verification when stubbing | ||
| locals in views. Details can be found in [this issue](https://github.com/rspec/rspec-mocks/issues/1102) |
There was a problem hiding this comment.
@myronmarston @samphippen's new wording explains the rspec-rails rational, we don't yet do this automatically although it could be added.
|
The screenshot needs to be recaptured since the color of summary line was changed in rspec/rspec-core#2421. |
Done:
|
|
@samphippen can you add a section about the better failure output for the |
I'll take a stab at this so we're ready to publish today. |
|
Also, I'm going to update the image again. |
|
OK, pushed my updates. |
|
OK. I think we've now addressed feedback and concerns. I'll do the stats and the rest of the post now! |
👍 If you want to make the publication date match the release date of 3.6, you'll need to update the filename once more. |
| `before(:suite)` hook is now formatted like this: | ||
|
|
||
| <img alt="Errors outside example execution" | ||
| src="/images/blog/errors_outside_example.png"> |
There was a problem hiding this comment.
Any reason why you're not using the markdown image syntax here?
@rspec/rspec I have a few open TODOs and need to generate the final stats, can I get a review?