From 620dcca25444ddea52868e7918d31de52d5cc8f6 Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Fri, 7 Apr 2017 14:59:19 +0200 Subject: [PATCH 01/20] Added the "Compatibility" chapter to the READMEs In order to make the support for different Elasticsearch versions more clear, a special "Compatibility" chapter has been added to the main README and individual gem READMEs. The versioning scheme follows the scheme for the Elasticsearch Ruby client, ie. the version number matches the supported major version of Elasticsearch. See: * https://github.com/elastic/elasticsearch-ruby/commit/3bdb385 --- README.md | 20 ++++++++++++++++---- elasticsearch-model/README.md | 16 ++++++++++++++-- elasticsearch-persistence/README.md | 14 +++++++++++++- elasticsearch-rails/README.md | 16 ++++++++++++++-- 4 files changed, 57 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 081508540..043677574 100644 --- a/README.md +++ b/README.md @@ -16,15 +16,27 @@ This repository contains various Ruby and Rails integrations for [Elasticsearch] Elasticsearch client and Ruby API is provided by the **[elasticsearch-ruby](https://github.com/elasticsearch/elasticsearch-ruby)** project. -## Installation +## Compatibility The libraries are compatible with Ruby 1.9.3 and higher. -Install the `elasticsearch-model` and/or `elasticsearch-rails` package from -[Rubygems](https://rubygems.org/gems/elasticsearch): +The version numbers follow the Elasticsearch major versions, and the `master` branch +is compatible with the Elasticsearch `master` branch, therefore, with the next major version. + +| Rubygem | | Elasticsearch | +|:-------------:|:-:| :-----------: | +| 0.1 | → | 1.x | +| 2.x | → | 2.x | +| 5.x | → | 5.x | +| master | → | master | + +## Installation + +Install each library from [Rubygems](https://rubygems.org/gems/elasticsearch): ```ruby -gem install elasticsearch-model elasticsearch-rails +gem install elasticsearch-model +gem install elasticsearch-rails ``` To use an unreleased version, either add it to your `Gemfile` for [Bundler](http://gembundler.com): diff --git a/elasticsearch-model/README.md b/elasticsearch-model/README.md index 3f5b8df74..b580e1c2a 100644 --- a/elasticsearch-model/README.md +++ b/elasticsearch-model/README.md @@ -1,13 +1,25 @@ # Elasticsearch::Model The `elasticsearch-model` library builds on top of the -the [`elasticsearch`](https://github.com/elasticsearch/elasticsearch-ruby) library. +the [`elasticsearch`](https://github.com/elastic/elasticsearch-ruby) library. It aims to simplify integration of Ruby classes ("models"), commonly found e.g. in [Ruby on Rails](http://rubyonrails.org) applications, with the [Elasticsearch](http://www.elasticsearch.org) search and analytics engine. -The library is compatible with Ruby 1.9.3 and higher. +## Compatibility + +This library is compatible with Ruby 1.9.3 and higher. + +The library version numbers follow the Elasticsearch major versions, and the `master` branch +is compatible with the Elasticsearch `master` branch, therefore, with the next major version. + +| Rubygem | | Elasticsearch | +|:-------------:|:-:| :-----------: | +| 0.1 | → | 1.x | +| 2.x | → | 2.x | +| 5.x | → | 5.x | +| master | → | master | ## Installation diff --git a/elasticsearch-persistence/README.md b/elasticsearch-persistence/README.md index 5a7921e09..d6b31f6d6 100644 --- a/elasticsearch-persistence/README.md +++ b/elasticsearch-persistence/README.md @@ -2,7 +2,19 @@ Persistence layer for Ruby domain objects in Elasticsearch, using the Repository and ActiveRecord patterns. -The library is compatible with Ruby 1.9.3 (or higher) and Elasticsearch 1.0 (or higher). +## Compatibility + +This library is compatible with Ruby 1.9.3 and higher. + +The library version numbers follow the Elasticsearch major versions, and the `master` branch +is compatible with the Elasticsearch `master` branch, therefore, with the next major version. + +| Rubygem | | Elasticsearch | +|:-------------:|:-:| :-----------: | +| 0.1 | → | 1.x | +| 2.x | → | 2.x | +| 5.x | → | 5.x | +| master | → | master | ## Installation diff --git a/elasticsearch-rails/README.md b/elasticsearch-rails/README.md index 476054938..ee513b5be 100644 --- a/elasticsearch-rails/README.md +++ b/elasticsearch-rails/README.md @@ -1,10 +1,22 @@ # Elasticsearch::Rails The `elasticsearch-rails` library is a companion for the -the [`elasticsearch-model`](https://github.com/elasticsearch/elasticsearch-rails/tree/master/elasticsearch-model) +the [`elasticsearch-model`](https://github.com/elastic/elasticsearch-rails/tree/master/elasticsearch-model) library, providing features suitable for Ruby on Rails applications. -The library is compatible with Ruby 1.9.3 and higher. +## Compatibility + +This library is compatible with Ruby 1.9.3 and higher. + +The library version numbers follow the Elasticsearch major versions, and the `master` branch +is compatible with the Elasticsearch `master` branch, therefore, with the next major version. + +| Rubygem | | Elasticsearch | +|:-------------:|:-:| :-----------: | +| 0.1 | → | 1.x | +| 2.x | → | 2.x | +| 5.x | → | 5.x | +| master | → | master | ## Installation From 05dd0e1529e63b197bd176e3ad094c737b3bf0c8 Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Fri, 7 Apr 2017 15:14:28 +0200 Subject: [PATCH 02/20] Updated the Bundler instructions and Github URLs in the READMEs --- README.md | 18 +++--------------- elasticsearch-model/README.md | 10 +++++----- elasticsearch-persistence/README.md | 12 ++++++------ elasticsearch-rails/README.md | 16 ++++++++-------- 4 files changed, 22 insertions(+), 34 deletions(-) diff --git a/README.md b/README.md index 043677574..f7c8e1151 100644 --- a/README.md +++ b/README.md @@ -39,23 +39,11 @@ gem install elasticsearch-model gem install elasticsearch-rails ``` -To use an unreleased version, either add it to your `Gemfile` for [Bundler](http://gembundler.com): +To use an unreleased version, add it to your `Gemfile` for [Bundler](http://bundler.io): ```ruby -gem 'elasticsearch-model', git: 'git://github.com/elasticsearch/elasticsearch-rails.git' -gem 'elasticsearch-rails', git: 'git://github.com/elasticsearch/elasticsearch-rails.git' -``` - -or install it from a source code checkout: - -```bash -git clone https://github.com/elasticsearch/elasticsearch-rails.git -cd elasticsearch-model -bundle install -rake install -cd elasticsearch-rails -bundle install -rake install +gem 'elasticsearch-model', github: 'elastic/elasticsearch-rails', branch: '5.x' +gem 'elasticsearch-rails', github: 'elastic/elasticsearch-rails', branch: '5.x' ``` ## Usage diff --git a/elasticsearch-model/README.md b/elasticsearch-model/README.md index b580e1c2a..1d06e8b89 100644 --- a/elasticsearch-model/README.md +++ b/elasticsearch-model/README.md @@ -29,11 +29,11 @@ Install the package from [Rubygems](https://rubygems.org): To use an unreleased version, either add it to your `Gemfile` for [Bundler](http://bundler.io): - gem 'elasticsearch-model', git: 'git://github.com/elasticsearch/elasticsearch-rails.git' + gem 'elasticsearch-model', git: 'git://github.com/elastic/elasticsearch-rails.git', branch: '5.x' or install it from a source code checkout: - git clone https://github.com/elasticsearch/elasticsearch-rails.git + git clone https://github.com/elastic/elasticsearch-rails.git cd elasticsearch-rails/elasticsearch-model bundle install rake install @@ -121,7 +121,7 @@ See the `Elasticsearch::Model` module documentation for technical information. ### The Elasticsearch client -The module will set up a [client](https://github.com/elasticsearch/elasticsearch-ruby/tree/master/elasticsearch), +The module will set up a [client](https://github.com/elastic/elasticsearch-ruby/tree/master/elasticsearch), connected to `localhost:9200`, by default. You can access and use it as any other `Elasticsearch::Client`: ```ruby @@ -144,7 +144,7 @@ Elasticsearch::Model.client = Elasticsearch::Client.new log: true You might want to do this during your application bootstrap process, e.g. in a Rails initializer. Please refer to the -[`elasticsearch-transport`](https://github.com/elasticsearch/elasticsearch-ruby/tree/master/elasticsearch-transport) +[`elasticsearch-transport`](https://github.com/elastic/elasticsearch-ruby/tree/master/elasticsearch-transport) library documentation for all the configuration options, and to the [`elasticsearch-api`](http://rubydoc.info/gems/elasticsearch-api) library documentation for information about the Ruby client API. @@ -253,7 +253,7 @@ response.records.order(:title).to_a The `records` method returns the real instances of your model, which is useful when you want to access your model methods -- at the expense of slowing down your application, of course. In most cases, working with `results` coming from Elasticsearch is sufficient, and much faster. See the -[`elasticsearch-rails`](https://github.com/elasticsearch/elasticsearch-rails/tree/master/elasticsearch-rails) +[`elasticsearch-rails`](https://github.com/elastic/elasticsearch-rails/tree/master/elasticsearch-rails) library for more information about compatibility with the Ruby on Rails framework. When you want to access both the database `records` and search `results`, use the `each_with_hit` diff --git a/elasticsearch-persistence/README.md b/elasticsearch-persistence/README.md index d6b31f6d6..8d02d227c 100644 --- a/elasticsearch-persistence/README.md +++ b/elasticsearch-persistence/README.md @@ -24,11 +24,11 @@ Install the package from [Rubygems](https://rubygems.org): To use an unreleased version, either add it to your `Gemfile` for [Bundler](http://bundler.io): - gem 'elasticsearch-persistence', git: 'git://github.com/elasticsearch/elasticsearch-rails.git' + gem 'elasticsearch-persistence', git: 'git://github.com/elastic/elasticsearch-rails.git', branch: '5.x' or install it from a source code checkout: - git clone https://github.com/elasticsearch/elasticsearch-rails.git + git clone https://github.com/elastic/elasticsearch-rails.git cd elasticsearch-rails/elasticsearch-persistence bundle install rake install @@ -112,7 +112,7 @@ repository.delete(note) The repository module provides a number of features and facilities to configure and customize the behavior: -* Configuring the Elasticsearch [client](https://github.com/elasticsearch/elasticsearch-ruby#usage) being used +* Configuring the Elasticsearch [client](https://github.com/elastic/elasticsearch-ruby#usage) being used * Setting the index name, document type, and object class for deserialization * Composing mappings and settings for the index * Creating, deleting or refreshing the index @@ -263,7 +263,7 @@ puts repository.find(1).attributes['image'] ##### Client -The repository uses the standard Elasticsearch [client](https://github.com/elasticsearch/elasticsearch-ruby#usage), +The repository uses the standard Elasticsearch [client](https://github.com/elastic/elasticsearch-ruby#usage), which is accessible with the `client` getter and setter methods: ```ruby @@ -622,7 +622,7 @@ puts results.response.aggregations.authors.buckets.each { |b| puts "#{b['key']} #### The Elasticsearch Client -The module will set up a [client](https://github.com/elasticsearch/elasticsearch-ruby/tree/master/elasticsearch), +The module will set up a [client](https://github.com/elastic/elasticsearch-ruby/tree/master/elasticsearch), connected to `localhost:9200`, by default. To use a client with different configuration: @@ -688,7 +688,7 @@ rails generate scaffold Person name:String email:String birthday:Date --orm=elas A fully working Ruby on Rails application can be generated with the following command: ```bash -rails new music --force --skip --skip-bundle --skip-active-record --template https://raw.githubusercontent.com/elasticsearch/elasticsearch-rails/master/elasticsearch-persistence/examples/music/template.rb +rails new music --force --skip --skip-bundle --skip-active-record --template https://raw.githubusercontent.com/elastic/elasticsearch-rails/master/elasticsearch-persistence/examples/music/template.rb ``` The application demonstrates: diff --git a/elasticsearch-rails/README.md b/elasticsearch-rails/README.md index ee513b5be..eb0cb2782 100644 --- a/elasticsearch-rails/README.md +++ b/elasticsearch-rails/README.md @@ -26,11 +26,11 @@ Install the package from [Rubygems](https://rubygems.org): To use an unreleased version, either add it to your `Gemfile` for [Bundler](http://bundler.io): - gem 'elasticsearch-rails', git: 'git://github.com/elasticsearch/elasticsearch-rails.git' + gem 'elasticsearch-rails', git: 'git://github.com/elastic/elasticsearch-rails.git', branch: '5.x' or install it from a source code checkout: - git clone https://github.com/elasticsearch/elasticsearch-rails.git + git clone https://github.com/elastic/elasticsearch-rails.git cd elasticsearch-rails/elasticsearch-rails bundle install rake install @@ -99,22 +99,22 @@ You should see the duration of the request to Elasticsearch as part of each log You can generate a fully working example Ruby on Rails application, with an `Article` model and a search form, to play with (it even downloads _Elasticsearch_ itself, generates the application skeleton and leaves you with a _Git_ repository to explore the steps and the code) with the -[`01-basic.rb`](https://github.com/elasticsearch/elasticsearch-rails/blob/master/elasticsearch-rails/lib/rails/templates/01-basic.rb) template: +[`01-basic.rb`](https://github.com/elastic/elasticsearch-rails/blob/master/elasticsearch-rails/lib/rails/templates/01-basic.rb) template: ```bash -rails new searchapp --skip --skip-bundle --template https://raw.github.com/elasticsearch/elasticsearch-rails/master/elasticsearch-rails/lib/rails/templates/01-basic.rb +rails new searchapp --skip --skip-bundle --template https://raw.github.com/elastic/elasticsearch-rails/master/elasticsearch-rails/lib/rails/templates/01-basic.rb ``` Run the same command again, in the same folder, with the -[`02-pretty`](https://github.com/elasticsearch/elasticsearch-rails/blob/master/elasticsearch-rails/lib/rails/templates/02-pretty.rb) +[`02-pretty`](https://github.com/elastic/elasticsearch-rails/blob/master/elasticsearch-rails/lib/rails/templates/02-pretty.rb) template to add features such as a custom `Article.search` method, result highlighting and [_Bootstrap_](http://getbootstrap.com) integration: ```bash -rails new searchapp --skip --skip-bundle --template https://raw.github.com/elasticsearch/elasticsearch-rails/master/elasticsearch-rails/lib/rails/templates/02-pretty.rb +rails new searchapp --skip --skip-bundle --template https://raw.github.com/elastic/elasticsearch-rails/master/elasticsearch-rails/lib/rails/templates/02-pretty.rb ``` -Run the same command with the [`03-expert.rb`](https://github.com/elasticsearch/elasticsearch-rails/blob/master/elasticsearch-rails/lib/rails/templates/03-expert.rb) +Run the same command with the [`03-expert.rb`](https://github.com/elastic/elasticsearch-rails/blob/master/elasticsearch-rails/lib/rails/templates/03-expert.rb) template to refactor the application into a more complex use case, with couple of hundreds of The New York Times articles as the example content. The template will extract the Elasticsearch integration into a `Searchable` "concern" module, @@ -122,7 +122,7 @@ define complex mapping, custom serialization, implement faceted navigation and s a complex query, and add a _Sidekiq_-based worker for updating the index in the background. ```bash -rails new searchapp --skip --skip-bundle --template https://raw.github.com/elasticsearch/elasticsearch-rails/master/elasticsearch-rails/lib/rails/templates/03-expert.rb +rails new searchapp --skip --skip-bundle --template https://raw.github.com/elastic/elasticsearch-rails/master/elasticsearch-rails/lib/rails/templates/03-expert.rb ``` ## License From 839da59e6b55476f2afad0009690919ca09c051b Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Fri, 7 Apr 2017 15:53:27 +0200 Subject: [PATCH 03/20] Added a CHANGELOG to the repository --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 CHANGELOG.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 000000000..ed56ccac5 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,4 @@ +## 0.1.9 + +The last version for the old versioning scheme -- please see the Git commit log +at https://github.com/elastic/elasticsearch-rails/commits/v0.1.9 From 83107f2886de9f859edbf694330d01ac4934a383 Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Fri, 7 Apr 2017 16:35:16 +0200 Subject: [PATCH 04/20] Release 5.0.0 --- CHANGELOG.md | 73 +++++++++++++++++++ .../lib/elasticsearch/model/version.rb | 2 +- .../lib/elasticsearch/persistence/version.rb | 2 +- .../lib/elasticsearch/rails/version.rb | 2 +- 4 files changed, 76 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ed56ccac5..3fee500ea 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,76 @@ +## 5.0.0 + +* Updated the Rake dependency to 11.1 +* Reduced verbosity of `rake test:unit` and `rake test:integration` +* Removed the "CI Reporter" integration from test Rake tasks +* Added the "Compatibility" chapter to the READMEs +* Updated the Bundler instructions and Github URLs in the READMEs + +### ActiveModel + +* Fixed a problem where `Hashie::Mash#min` and `#max` returned unexpected values +* Added information about `elasticsearch-dsl` to the README +* Added support for inherited index names and doc types +* Added a `Elasticsearch::Model.settings` method +* Changed the naming inheritance logic to use `Elasticsearch::Model.settings` +* Added information about the `settings` method and the `inheritance_enabled` setting into the README +* Disable "verbose" and "warnings" in integration tests +* Added code for establishing ActiveRecord connections to test classes +* Reorganized the class definitions in the integration tests +* Moved `require` within unit test to the top of the file +* Added ActiveRecord 5 support to integration test configuration +* Fixed records sorting with ActiveRecord 5.x +* Added, that `add_index` for ActiveRecord models is only called when it doesn't exist already +* Use `records.__send__ :load` instead of `records.load` in the ActiveRecord adapter +* Call `Kaminari::Hooks.init` only when available +* Fixed the deprecation messages for `raise_in_transactional_callbacks` +* Fixed the deprecation messages for `timestamps` in migrations in integration tests +* Fixed the naming for the indexing integration tests +* Fixed the failing integration tests for ActiveRecord associations +* Fixed integration tests for ActiveRecord pagination +* Added the `rake bundle:install` Rake task to install dependencies for all gemfiles +* Run unit tests against all Gemfiles +* Updated dependencies in gemspec +* Relaxed the dependency on the "elasticsearch" gem +* Fixed the completion example for ActiveRecord for Elasticsearch 5 +* Added an example with Edge NGram mapping for auto-completion +* Expanded the example for indexing and searching ActiveRecord associations +* Added an example for source filtering to the ActiveRecord associations example +* Fixed a typo in the README +* Changed the default mapping type to `text` +* Added a `HashWrapper` class to wrap Hash structures instead of raw `Hashie::Mash` +* Call `Hashie.disable_warnings` method in Response wrappers +* Added, that `HashWrapper`, a sub-class of `Hashie::Mash` is used +* Updated the configuration for required routing in the integration test +* Fixed incorrect name for the parent/child integration test +* Fixed incorrect mapping configuration in the integration tests +* Allow passing the index settings and mappings as arguments to `create_index!` +* Added instructions about creating the index into the README +* Updated the "completion suggester" example + +### Persistence + +* Updated dependencies in gemspec +* Updated dependencies in gemspec +* Relaxed the dependency on the "elasticsearch" gem +* Use `text` instead of `string` for the data types +* Changed the default mapping type to `text` +* Removed the `search_type=scan` in the `find_in_batches` method +* Updated the `count` method in the "repository" module +* Updated the "update by script" integration test for Elasticsearch 5 +* Added, that `HashWrapper`, a sub-class of `Hashie::Mash` is used +* Updated the "Notes" example application for Elasticsearch 5.x +* Updated the "Music" example application for Elasticsearch 5.x +* Updated the URLs in the "Music" application template +* Updated the Git URLs in the "Notes" example application + +### Ruby on Rails + +* Updated the application templates to support Rails 5 & Elasticsearch 5 +* Updated the `03-expert` application template to work with Rails 5 +* Updated the application templates to work with README.md instead of README.rdoc +* Updated the installation process in the "01-basic" application template + ## 0.1.9 The last version for the old versioning scheme -- please see the Git commit log diff --git a/elasticsearch-model/lib/elasticsearch/model/version.rb b/elasticsearch-model/lib/elasticsearch/model/version.rb index 44cfdabea..d64e61b61 100644 --- a/elasticsearch-model/lib/elasticsearch/model/version.rb +++ b/elasticsearch-model/lib/elasticsearch/model/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Model - VERSION = "0.1.9" + VERSION = "5.0.0" end end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb index 93852c314..dfa08b666 100644 --- a/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Persistence - VERSION = "0.1.9" + VERSION = "5.0.0" end end diff --git a/elasticsearch-rails/lib/elasticsearch/rails/version.rb b/elasticsearch-rails/lib/elasticsearch/rails/version.rb index 88b2dd758..4dae8ed25 100644 --- a/elasticsearch-rails/lib/elasticsearch/rails/version.rb +++ b/elasticsearch-rails/lib/elasticsearch/rails/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Rails - VERSION = "0.1.9" + VERSION = "5.0.0" end end From c24302686879bfb6d7ef5eecb745482a6892ba4a Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Fri, 7 Apr 2017 16:57:39 +0200 Subject: [PATCH 05/20] Updated the version of Elasticsearch to run at Travis to "5.3.0" --- .travis.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 364a58c3b..07aca469c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -15,6 +15,7 @@ branches: only: - master - travis + - 5.x matrix: include: @@ -28,14 +29,14 @@ matrix: - rvm: 2.3.3 jdk: oraclejdk8 - env: TEST_SUITE=integration QUIET=y SERVER=start TEST_CLUSTER_LOGS=/tmp/log TEST_CLUSTER_COMMAND=/tmp/elasticsearch-5.2.0/bin/elasticsearch + env: TEST_SUITE=integration QUIET=y SERVER=start TEST_CLUSTER_LOGS=/tmp/log TEST_CLUSTER_COMMAND=/tmp/elasticsearch-5.3.0/bin/elasticsearch before_install: - gem update --system --no-rdoc --no-ri - gem --version - gem install bundler -v 1.14.3 --no-rdoc --no-ri - bundle version - - curl -sS https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.2.0.tar.gz | tar xz -C /tmp + - curl -sS https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.3.0.tar.gz | tar xz -C /tmp install: - bundle install From d1d4dd97bb0f60175b7e6f2176e63e8f50eb1620 Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Mon, 10 Apr 2017 16:36:49 +0200 Subject: [PATCH 06/20] [STORE] Updated the failing integration tests for Elasticsearch 5.x Related: #693 --- elasticsearch-persistence/README.md | 4 ++-- .../lib/elasticsearch/persistence/model/find.rb | 3 ++- .../test/integration/model/model_basic_test.rb | 4 ++-- elasticsearch-persistence/test/unit/model_rails_test.rb | 2 +- 4 files changed, 7 insertions(+), 6 deletions(-) diff --git a/elasticsearch-persistence/README.md b/elasticsearch-persistence/README.md index 8d02d227c..4c6dead10 100644 --- a/elasticsearch-persistence/README.md +++ b/elasticsearch-persistence/README.md @@ -491,8 +491,8 @@ class Article # Define an `author` attribute, with multiple analyzers for this field # attribute :author, String, mapping: { fields: { - author: { type: 'string'}, - raw: { type: 'string', analyzer: 'keyword' } + author: { type: 'text'}, + raw: { type: 'keyword' } } } diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/model/find.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/model/find.rb index 43276dc03..c09c9ac38 100644 --- a/elasticsearch-persistence/lib/elasticsearch/persistence/model/find.rb +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/model/find.rb @@ -119,7 +119,8 @@ def find_in_batches(options={}, &block) # Get the initial batch of documents and the scroll_id # - response = gateway.client.search( { index: gateway.index_name, + response = gateway.client.search( { + index: gateway.index_name, type: gateway.document_type, scroll: scroll, sort: ['_doc'], diff --git a/elasticsearch-persistence/test/integration/model/model_basic_test.rb b/elasticsearch-persistence/test/integration/model/model_basic_test.rb index b052114f8..33555c980 100644 --- a/elasticsearch-persistence/test/integration/model/model_basic_test.rb +++ b/elasticsearch-persistence/test/integration/model/model_basic_test.rb @@ -16,8 +16,8 @@ class ::Person attribute :name, String, mapping: { fields: { - name: { type: 'string', analyzer: 'snowball' }, - raw: { type: 'string', analyzer: 'keyword' } + name: { type: 'text', analyzer: 'snowball' }, + raw: { type: 'keyword' } } } attribute :birthday, Date diff --git a/elasticsearch-persistence/test/unit/model_rails_test.rb b/elasticsearch-persistence/test/unit/model_rails_test.rb index 9ead42896..3f0ce913b 100644 --- a/elasticsearch-persistence/test/unit/model_rails_test.rb +++ b/elasticsearch-persistence/test/unit/model_rails_test.rb @@ -11,7 +11,7 @@ class ::MyRailsModel include Elasticsearch::Persistence::Model include Elasticsearch::Persistence::Model::Rails - attribute :name, String, mapping: { analyzer: 'string' } + attribute :name, String, mapping: { analyzer: 'english' } attribute :published_at, DateTime attribute :published_on, Date end From 2010cf23c0251969d5d0423c5935497738eb4f6b Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Mon, 10 Apr 2017 16:43:36 +0200 Subject: [PATCH 07/20] [STORE] Updated the dependency for "elasticsearch" and "elasticsearch-model" to `5.x` Related: #693 --- elasticsearch-persistence/elasticsearch-persistence.gemspec | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/elasticsearch-persistence/elasticsearch-persistence.gemspec b/elasticsearch-persistence/elasticsearch-persistence.gemspec index 510da3a3e..311325fdc 100644 --- a/elasticsearch-persistence/elasticsearch-persistence.gemspec +++ b/elasticsearch-persistence/elasticsearch-persistence.gemspec @@ -23,8 +23,8 @@ Gem::Specification.new do |s| s.required_ruby_version = ">= 1.9.3" - s.add_dependency "elasticsearch", '> 1' - s.add_dependency "elasticsearch-model", '>= 0.1' + s.add_dependency "elasticsearch", '~> 5' + s.add_dependency "elasticsearch-model", '~> 5' s.add_dependency "activesupport", '> 4' s.add_dependency "activemodel", '> 4' s.add_dependency "hashie" From bce6c42a873135b81876288fa0263f6299092cb7 Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Thu, 4 May 2017 15:45:35 +0200 Subject: [PATCH 08/20] [MODEL] Updated the dependency on "elasticsearch" to 5.x The 5.x versions should depend on the corresponding version of the Ruby client. Related: #702 --- elasticsearch-model/elasticsearch-model.gemspec | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/elasticsearch-model/elasticsearch-model.gemspec b/elasticsearch-model/elasticsearch-model.gemspec index e058ac43d..ab00793cf 100644 --- a/elasticsearch-model/elasticsearch-model.gemspec +++ b/elasticsearch-model/elasticsearch-model.gemspec @@ -23,7 +23,7 @@ Gem::Specification.new do |s| s.required_ruby_version = ">= 1.9.3" - s.add_dependency "elasticsearch", '> 1' + s.add_dependency "elasticsearch", '~> 5' s.add_dependency "activesupport", '> 3' s.add_dependency "hashie" From ccf2e8e08ceeb0fa1f1ca1facb5943cfb3a495c3 Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Fri, 7 Apr 2017 16:46:08 +0200 Subject: [PATCH 09/20] Added a Rake task for updating the version and compiling the CHANGELOG --- Rakefile | 114 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/Rakefile b/Rakefile index 64fab6ac6..b4c90d710 100644 --- a/Rakefile +++ b/Rakefile @@ -126,3 +126,117 @@ task :release do puts '-'*80 end end + +desc <<-DESC + Update Rubygems versions in version.rb and *.gemspec files + + Example: + + $ rake update_version[5.0.0,5.0.1] +DESC +task :update_version, :old, :new do |task, args| + require 'ansi' + + puts "[!!!] Required argument [old] missing".ansi(:red) unless args[:old] + puts "[!!!] Required argument [new] missing".ansi(:red) unless args[:new] + + files = Dir['**/**/version.rb','**/**/*.gemspec'] + + longest_line = files.map { |f| f.size }.max + + puts "\n", "= FILES ".ansi(:faint) + ('='*92).ansi(:faint), "\n" + + files.each do |file| + begin + File.open(file, 'r+') do |f| + content = f.read + if content.match Regexp.new(args[:old]) + content.gsub! Regexp.new(args[:old]), args[:new] + puts "+ [#{file}]".ansi(:green).ljust(longest_line+20) + " [#{args[:old]}] -> [#{args[:new]}]".ansi(:green,:bold) + f.rewind + f.write content + else + puts "- [#{file}]".ansi(:yellow).ljust(longest_line+20) + " -".ansi(:faint,:strike) + end + end + rescue Exception => e + puts "[!!!] #{e.class} : #{e.message}".ansi(:red,:bold) + raise e + end + end + + puts "\n\n", "= CHANGELOG ".ansi(:faint) + ('='*88).ansi(:faint), "\n" + + log = `git --no-pager log --reverse --no-color --pretty='* %s' HEAD --not v#{args[:old]} elasticsearch-*`.split("\n") + + puts log.join("\n") + + log_entries = {} + log_entries[:common] = log.reject { |l| l =~ /^* \[/ } + log_entries[:model] = log.select { |l| l =~ /^* \[MODEL\]/ } + log_entries[:store] = log.select { |l| l =~ /^* \[STORE\]/ } + log_entries[:rails] = log.select { |l| l =~ /^* \[RAILS\]/ } + + changelog = File.read(File.open('CHANGELOG.md', 'r')) + + changelog_update = '' + + changelog_update << "## #{args[:new]}\n\n" + + unless log_entries[:common].empty? + changelog_update << log_entries[:common] + .map { |l| "#{l}" } + .join("\n") + changelog_update << "\n\n" + end + + unless log_entries[:model].empty? + changelog_update << "### ActiveModel\n\n" + changelog_update << log_entries[:model] + .map { |l| l.gsub /\[.+\] /, '' } + .map { |l| "#{l}" } + .join("\n") + changelog_update << "\n\n" + end + + unless log_entries[:store].empty? + changelog_update << "### Persistence\n\n" + changelog_update << log_entries[:store] + .map { |l| l.gsub /\[.+\] /, '' } + .map { |l| "#{l}" } + .join("\n") + changelog_update << "\n\n" + end + + unless log_entries[:rails].empty? + changelog_update << "### Ruby on Rails\n\n" + changelog_update << log_entries[:rails] + .map { |l| l.gsub /\[.+\] /, '' } + .map { |l| "#{l}" } + .join("\n") + changelog_update << "\n\n" + end + + unless changelog =~ /^## #{args[:new]}/ + File.open('CHANGELOG.md', 'w+') { |f| f.write changelog_update and f.write changelog } + end + + puts "\n\n", "= DIFF ".ansi(:faint) + ('='*93).ansi(:faint) + + diff = `git --no-pager diff --patch --word-diff=color --minimal elasticsearch-*`.split("\n") + + puts diff + .reject { |l| l =~ /^\e\[1mdiff \-\-git/ } + .reject { |l| l =~ /^\e\[1mindex [a-z0-9]{7}/ } + .reject { |l| l =~ /^\e\[1m\-\-\- i/ } + .reject { |l| l =~ /^\e\[36m@@/ } + .map { |l| l =~ /^\e\[1m\+\+\+ w/ ? "\n#{l} " + '-'*(104-l.size) : l } + .join("\n") + + puts "\n\n", "= COMMIT ".ansi(:faint) + ('='*91).ansi(:faint), "\n" + + puts "git add CHANGELOG.md elasticsearch-*", + "git commit --verbose --message='Release #{args[:new]}' --edit", + "rake release" + "\n" +end From 22b698aa3f4138db6ec1d1a082e8cd24f8132224 Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Thu, 4 May 2017 15:51:13 +0200 Subject: [PATCH 10/20] Release 5.0.1 --- CHANGELOG.md | 11 +++++++++++ .../lib/elasticsearch/model/version.rb | 2 +- .../lib/elasticsearch/persistence/version.rb | 2 +- .../lib/elasticsearch/rails/version.rb | 2 +- 4 files changed, 14 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3fee500ea..bed561748 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,14 @@ +## 5.0.1 + +### ActiveModel + +* Updated the dependency on "elasticsearch" to 5.x + +### Persistence + +* Updated the failing integration tests for Elasticsearch 5.x +* Updated the dependency for "elasticsearch" and "elasticsearch-model" to `5.x` + ## 5.0.0 * Updated the Rake dependency to 11.1 diff --git a/elasticsearch-model/lib/elasticsearch/model/version.rb b/elasticsearch-model/lib/elasticsearch/model/version.rb index d64e61b61..14f022513 100644 --- a/elasticsearch-model/lib/elasticsearch/model/version.rb +++ b/elasticsearch-model/lib/elasticsearch/model/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Model - VERSION = "5.0.0" + VERSION = "5.0.1" end end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb index dfa08b666..c8e1f2b1f 100644 --- a/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Persistence - VERSION = "5.0.0" + VERSION = "5.0.1" end end diff --git a/elasticsearch-rails/lib/elasticsearch/rails/version.rb b/elasticsearch-rails/lib/elasticsearch/rails/version.rb index 4dae8ed25..e2fc23ad4 100644 --- a/elasticsearch-rails/lib/elasticsearch/rails/version.rb +++ b/elasticsearch-rails/lib/elasticsearch/rails/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Rails - VERSION = "5.0.0" + VERSION = "5.0.1" end end From 900899cd7ba017e54be519f637dce56e597289c1 Mon Sep 17 00:00:00 2001 From: Josh Becker Date: Tue, 28 Nov 2017 15:09:27 -0500 Subject: [PATCH 11/20] [MODEL] Updated the `changes` method name in `Indexing` to `changes_to_save` for compatibility with Rails 5.1 Without this patch, the log on Rails 5.1 is full of deprecation warnings like: DEPRECATION WARNING: The behavior of `changed_attributes` inside of after callbacks will be changing in the next version of Rails. The new return value will reflect the behavior of calling the method after `save` returned (e.g. the opposite of what it returns now). To maintain the current behavior, use `saved_changes.transform_values(&:first)` instead. (called from update_funding_caches at... It was first reported in elastic/elasticsearch-rails#714. This patch fixes #758. --- .../lib/elasticsearch/model/indexing.rb | 14 ++++++------ .../lib/elasticsearch/model/proxy.rb | 10 ++++----- .../test/unit/indexing_test.rb | 22 +++++++++---------- elasticsearch-model/test/unit/proxy_test.rb | 8 +++---- 4 files changed, 27 insertions(+), 27 deletions(-) diff --git a/elasticsearch-model/lib/elasticsearch/model/indexing.rb b/elasticsearch-model/lib/elasticsearch/model/indexing.rb index ac507b079..d58472a9f 100644 --- a/elasticsearch-model/lib/elasticsearch/model/indexing.rb +++ b/elasticsearch-model/lib/elasticsearch/model/indexing.rb @@ -307,7 +307,7 @@ module InstanceMethods def self.included(base) # Register callback for storing changed attributes for models - # which implement `before_save` and `changed_attributes` methods + # which implement `before_save` and `attributes_in_database` methods # # @note This is typically triggered only when the module would be # included in the model directly, not within the proxy. @@ -315,9 +315,9 @@ def self.included(base) # @see #update_document # base.before_save do |instance| - instance.instance_variable_set(:@__changed_attributes, - Hash[ instance.changes.map { |key, value| [key, value.last] } ]) - end if base.respond_to?(:before_save) && base.instance_methods.include?(:changed_attributes) + instance.instance_variable_set(:@__attributes_in_database, + Hash[ instance.changes_to_save.map { |key, value| [key, value.last] } ]) + end if base.respond_to?(:before_save) && base.instance_methods.include?(:attributes_in_database) end # Serializes the model instance into JSON (by calling `as_indexed_json`), @@ -391,11 +391,11 @@ def delete_document(options={}) # @see http://rubydoc.info/gems/elasticsearch-api/Elasticsearch/API/Actions:update # def update_document(options={}) - if changed_attributes = self.instance_variable_get(:@__changed_attributes) + if attributes_in_database = self.instance_variable_get(:@__attributes_in_database) attributes = if respond_to?(:as_indexed_json) - self.as_indexed_json.select { |k,v| changed_attributes.keys.map(&:to_s).include? k.to_s } + self.as_indexed_json.select { |k,v| attributes_in_database.keys.map(&:to_s).include? k.to_s } else - changed_attributes + attributes_in_database end client.update( diff --git a/elasticsearch-model/lib/elasticsearch/model/proxy.rb b/elasticsearch-model/lib/elasticsearch/model/proxy.rb index 3e37f28ec..606c079a6 100644 --- a/elasticsearch-model/lib/elasticsearch/model/proxy.rb +++ b/elasticsearch-model/lib/elasticsearch/model/proxy.rb @@ -54,15 +54,15 @@ def __elasticsearch__ &block end # Register a callback for storing changed attributes for models which implement - # `before_save` and `changed_attributes` methods (when `Elasticsearch::Model` is included) + # `before_save` and `attributes_in_database` methods (when `Elasticsearch::Model` is included) # # @see http://api.rubyonrails.org/classes/ActiveModel/Dirty.html # before_save do |i| - changed_attr = i.__elasticsearch__.instance_variable_get(:@__changed_attributes) || {} - i.__elasticsearch__.instance_variable_set(:@__changed_attributes, - changed_attr.merge(Hash[ i.changes.map { |key, value| [key, value.last] } ])) - end if respond_to?(:before_save) && instance_methods.include?(:changed_attributes) + changed_attr = i.__elasticsearch__.instance_variable_get(:@__attributes_in_database) || {} + i.__elasticsearch__.instance_variable_set(:@__attributes_in_database, + changed_attr.merge(Hash[ i.changes_to_save.map { |key, value| [key, value.last] } ])) + end if respond_to?(:before_save) && instance_methods.include?(:attributes_in_database) end end diff --git a/elasticsearch-model/test/unit/indexing_test.rb b/elasticsearch-model/test/unit/indexing_test.rb index b78651340..94f3d967b 100644 --- a/elasticsearch-model/test/unit/indexing_test.rb +++ b/elasticsearch-model/test/unit/indexing_test.rb @@ -171,9 +171,9 @@ def self.before_save(&block) (@callbacks ||= {})[block.hash] = block end - def changed_attributes; [:foo]; end + def attributes_in_database; [:foo]; end - def changes + def changes_to_save {:foo => ['One', 'Two']} end end @@ -186,9 +186,9 @@ def self.before_save(&block) (@callbacks ||= {})[block.hash] = block end - def changed_attributes; [:foo, :bar]; end + def attributes_in_database; [:foo, :bar]; end - def changes + def changes_to_save {:foo => ['A', 'B'], :bar => ['C', 'D']} end @@ -202,10 +202,10 @@ def as_indexed_json(options={}) ::DummyIndexingModelWithCallbacks.__send__ :include, Elasticsearch::Model::Indexing::InstanceMethods end - should "set the @__changed_attributes variable before save" do + should "set the @__attributes_in_database variable before save" do instance = ::DummyIndexingModelWithCallbacks.new instance.expects(:instance_variable_set).with do |name, value| - assert_equal :@__changed_attributes, name + assert_equal :@__attributes_in_database, name assert_equal({foo: 'Two'}, value) true end @@ -297,7 +297,7 @@ def as_indexed_json(options={}) instance = ::DummyIndexingModelWithCallbacks.new # Reset the fake `changes` - instance.instance_variable_set(:@__changed_attributes, nil) + instance.instance_variable_set(:@__attributes_in_database, nil) instance.expects(:index_document) instance.update_document @@ -308,7 +308,7 @@ def as_indexed_json(options={}) instance = ::DummyIndexingModelWithCallbacks.new # Set the fake `changes` hash - instance.instance_variable_set(:@__changed_attributes, {foo: 'bar'}) + instance.instance_variable_set(:@__attributes_in_database, {foo: 'bar'}) client.expects(:update).with do |payload| assert_equal 'foo', payload[:index] @@ -331,7 +331,7 @@ def as_indexed_json(options={}) instance = ::DummyIndexingModelWithCallbacksAndCustomAsIndexedJson.new # Set the fake `changes` hash - instance.instance_variable_set(:@__changed_attributes, {'foo' => 'B', 'bar' => 'D' }) + instance.instance_variable_set(:@__attributes_in_database, {'foo' => 'B', 'bar' => 'D' }) client.expects(:update).with do |payload| assert_equal({:foo => 'B'}, payload[:body][:doc]) @@ -350,7 +350,7 @@ def as_indexed_json(options={}) client = mock('client') instance = ::DummyIndexingModelWithCallbacksAndCustomAsIndexedJson.new - instance.instance_variable_set(:@__changed_attributes, { 'foo' => { 'bar' => 'BAR'} }) + instance.instance_variable_set(:@__attributes_in_database, { 'foo' => { 'bar' => 'BAR'} }) # Overload as_indexed_json instance.expects(:as_indexed_json).returns({ 'foo' => 'BAR' }) @@ -372,7 +372,7 @@ def as_indexed_json(options={}) instance = ::DummyIndexingModelWithCallbacks.new # Set the fake `changes` hash - instance.instance_variable_set(:@__changed_attributes, {author: 'john'}) + instance.instance_variable_set(:@__attributes_in_database, {author: 'john'}) client.expects(:update).with do |payload| assert_equal 'foo', payload[:index] diff --git a/elasticsearch-model/test/unit/proxy_test.rb b/elasticsearch-model/test/unit/proxy_test.rb index d7299f884..e403b6141 100644 --- a/elasticsearch-model/test/unit/proxy_test.rb +++ b/elasticsearch-model/test/unit/proxy_test.rb @@ -23,9 +23,9 @@ def self.before_save(&block) (@callbacks ||= {})[block.hash] = block end - def changed_attributes; [:foo]; end + def attributes_in_database; [:foo]; end - def changes + def changes_to_save {:foo => ['One', 'Two']} end end @@ -43,10 +43,10 @@ def changes DummyProxyModelWithCallbacks.__send__ :include, Elasticsearch::Model::Proxy end - should "set the @__changed_attributes variable before save" do + should "set the @__attributes_in_database variable before save" do instance = ::DummyProxyModelWithCallbacks.new instance.__elasticsearch__.expects(:instance_variable_set).with do |name, value| - assert_equal :@__changed_attributes, name + assert_equal :@__attributes_in_database, name assert_equal({foo: 'Two'}, value) true end From a41d9a291530b3eb69b3879b6897127263ff5990 Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Mon, 4 Dec 2017 17:41:21 +0100 Subject: [PATCH 12/20] [MODEL] Fixed the handling of changed attributes in `Indexing` to work with older Rails versions This patch builds on work in #738 by @jkeam and #758 by @Geesu to prevent deprecation warnings on Rails 5.1 due to changes in the handling of "changed attributes", originally reported by @davedash in #714. It's primary focus is to restore the compatibility with older Rails versions (so we don't break compatibility without proper version bump and in a single isolated commit), and to make the naming a bit more descriptive. First, the condition has been changed to work with the `changes_to_save` and `changes` methods, as opposed to the original `changed_attributes` / `attributes_in_database` naming. This communicates much more clearly the intent, and the original usage of `changed_attributes` has been misleading. Also, the "internal instance variable" which keeps track of the changes has been renamed to `@__changed_model_attributes`, in order to cleary differentiate it from the `changed_attributes` method name in older ActiveRecord versions. Closes #758 --- .../lib/elasticsearch/model/indexing.rb | 18 ++++--- .../lib/elasticsearch/model/proxy.rb | 16 +++++-- .../test/unit/indexing_test.rb | 48 ++++++++++++++----- elasticsearch-model/test/unit/proxy_test.rb | 6 +-- 4 files changed, 62 insertions(+), 26 deletions(-) diff --git a/elasticsearch-model/lib/elasticsearch/model/indexing.rb b/elasticsearch-model/lib/elasticsearch/model/indexing.rb index d58472a9f..39ad06bc3 100644 --- a/elasticsearch-model/lib/elasticsearch/model/indexing.rb +++ b/elasticsearch-model/lib/elasticsearch/model/indexing.rb @@ -307,17 +307,23 @@ module InstanceMethods def self.included(base) # Register callback for storing changed attributes for models - # which implement `before_save` and `attributes_in_database` methods + # which implement `before_save` and return changed attributes + # (ie. when `Elasticsearch::Model` is included) # # @note This is typically triggered only when the module would be # included in the model directly, not within the proxy. # # @see #update_document # - base.before_save do |instance| - instance.instance_variable_set(:@__attributes_in_database, - Hash[ instance.changes_to_save.map { |key, value| [key, value.last] } ]) - end if base.respond_to?(:before_save) && base.instance_methods.include?(:attributes_in_database) + base.before_save do |i| + if i.class.instance_methods.include?(:changes_to_save) # Rails 5.1 + i.instance_variable_set(:@__changed_model_attributes, + Hash[ i.changes_to_save.map { |key, value| [key, value.last] } ]) + elsif i.class.instance_methods.include?(:changes) + i.instance_variable_set(:@__changed_model_attributes, + Hash[ i.changes.map { |key, value| [key, value.last] } ]) + end + end if base.respond_to?(:before_save) end # Serializes the model instance into JSON (by calling `as_indexed_json`), @@ -391,7 +397,7 @@ def delete_document(options={}) # @see http://rubydoc.info/gems/elasticsearch-api/Elasticsearch/API/Actions:update # def update_document(options={}) - if attributes_in_database = self.instance_variable_get(:@__attributes_in_database) + if attributes_in_database = self.instance_variable_get(:@__changed_model_attributes) attributes = if respond_to?(:as_indexed_json) self.as_indexed_json.select { |k,v| attributes_in_database.keys.map(&:to_s).include? k.to_s } else diff --git a/elasticsearch-model/lib/elasticsearch/model/proxy.rb b/elasticsearch-model/lib/elasticsearch/model/proxy.rb index 606c079a6..a1c89ca95 100644 --- a/elasticsearch-model/lib/elasticsearch/model/proxy.rb +++ b/elasticsearch-model/lib/elasticsearch/model/proxy.rb @@ -54,15 +54,21 @@ def __elasticsearch__ &block end # Register a callback for storing changed attributes for models which implement - # `before_save` and `attributes_in_database` methods (when `Elasticsearch::Model` is included) + # `before_save` method and return changed attributes (ie. when `Elasticsearch::Model` is included) # # @see http://api.rubyonrails.org/classes/ActiveModel/Dirty.html # before_save do |i| - changed_attr = i.__elasticsearch__.instance_variable_get(:@__attributes_in_database) || {} - i.__elasticsearch__.instance_variable_set(:@__attributes_in_database, - changed_attr.merge(Hash[ i.changes_to_save.map { |key, value| [key, value.last] } ])) - end if respond_to?(:before_save) && instance_methods.include?(:attributes_in_database) + if i.class.instance_methods.include?(:changes_to_save) # Rails 5.1 + a = i.__elasticsearch__.instance_variable_get(:@__changed_model_attributes) || {} + i.__elasticsearch__.instance_variable_set(:@__changed_model_attributes, + a.merge(Hash[ i.changes_to_save.map { |key, value| [key, value.last] } ])) + elsif i.class.instance_methods.include?(:changes) + a = i.__elasticsearch__.instance_variable_get(:@__changed_model_attributes) || {} + i.__elasticsearch__.instance_variable_set(:@__changed_model_attributes, + a.merge(Hash[ i.changes.map { |key, value| [key, value.last] } ])) + end + end if respond_to?(:before_save) end end diff --git a/elasticsearch-model/test/unit/indexing_test.rb b/elasticsearch-model/test/unit/indexing_test.rb index 94f3d967b..e3bfad868 100644 --- a/elasticsearch-model/test/unit/indexing_test.rb +++ b/elasticsearch-model/test/unit/indexing_test.rb @@ -171,8 +171,6 @@ def self.before_save(&block) (@callbacks ||= {})[block.hash] = block end - def attributes_in_database; [:foo]; end - def changes_to_save {:foo => ['One', 'Two']} end @@ -186,8 +184,6 @@ def self.before_save(&block) (@callbacks ||= {})[block.hash] = block end - def attributes_in_database; [:foo, :bar]; end - def changes_to_save {:foo => ['A', 'B'], :bar => ['C', 'D']} end @@ -197,15 +193,28 @@ def as_indexed_json(options={}) end end + class ::DummyIndexingModelWithOldDirty + extend Elasticsearch::Model::Indexing::ClassMethods + include Elasticsearch::Model::Indexing::InstanceMethods + + def self.before_save(&block) + (@callbacks ||= {})[block.hash] = block + end + + def changes + {:foo => ['One', 'Two']} + end + end + should "register before_save callback when included" do ::DummyIndexingModelWithCallbacks.expects(:before_save).returns(true) ::DummyIndexingModelWithCallbacks.__send__ :include, Elasticsearch::Model::Indexing::InstanceMethods end - should "set the @__attributes_in_database variable before save" do + should "set the @__changed_model_attributes variable before save" do instance = ::DummyIndexingModelWithCallbacks.new instance.expects(:instance_variable_set).with do |name, value| - assert_equal :@__attributes_in_database, name + assert_equal :@__changed_model_attributes, name assert_equal({foo: 'Two'}, value) true end @@ -217,6 +226,23 @@ def as_indexed_json(options={}) end end + # https://github.com/elastic/elasticsearch-rails/issues/714 + # https://github.com/rails/rails/pull/25337#issuecomment-225166796 + should "set the @__changed_model_attributes variable before save for old ActiveModel::Dirty" do + instance = ::DummyIndexingModelWithOldDirty.new + instance.expects(:instance_variable_set).with do |name, value| + assert_equal :@__changed_model_attributes, name + assert_equal({foo: 'Two'}, value) + true + end + + ::DummyIndexingModelWithOldDirty.__send__ :include, Elasticsearch::Model::Indexing::InstanceMethods + + ::DummyIndexingModelWithOldDirty.instance_variable_get(:@callbacks).each do |n,b| + instance.instance_eval(&b) + end + end + should "have the index_document method" do client = mock('client') instance = ::DummyIndexingModelWithCallbacks.new @@ -297,7 +323,7 @@ def as_indexed_json(options={}) instance = ::DummyIndexingModelWithCallbacks.new # Reset the fake `changes` - instance.instance_variable_set(:@__attributes_in_database, nil) + instance.instance_variable_set(:@__changed_model_attributes, nil) instance.expects(:index_document) instance.update_document @@ -308,7 +334,7 @@ def as_indexed_json(options={}) instance = ::DummyIndexingModelWithCallbacks.new # Set the fake `changes` hash - instance.instance_variable_set(:@__attributes_in_database, {foo: 'bar'}) + instance.instance_variable_set(:@__changed_model_attributes, {foo: 'bar'}) client.expects(:update).with do |payload| assert_equal 'foo', payload[:index] @@ -331,7 +357,7 @@ def as_indexed_json(options={}) instance = ::DummyIndexingModelWithCallbacksAndCustomAsIndexedJson.new # Set the fake `changes` hash - instance.instance_variable_set(:@__attributes_in_database, {'foo' => 'B', 'bar' => 'D' }) + instance.instance_variable_set(:@__changed_model_attributes, {'foo' => 'B', 'bar' => 'D' }) client.expects(:update).with do |payload| assert_equal({:foo => 'B'}, payload[:body][:doc]) @@ -350,7 +376,7 @@ def as_indexed_json(options={}) client = mock('client') instance = ::DummyIndexingModelWithCallbacksAndCustomAsIndexedJson.new - instance.instance_variable_set(:@__attributes_in_database, { 'foo' => { 'bar' => 'BAR'} }) + instance.instance_variable_set(:@__changed_model_attributes, { 'foo' => { 'bar' => 'BAR'} }) # Overload as_indexed_json instance.expects(:as_indexed_json).returns({ 'foo' => 'BAR' }) @@ -372,7 +398,7 @@ def as_indexed_json(options={}) instance = ::DummyIndexingModelWithCallbacks.new # Set the fake `changes` hash - instance.instance_variable_set(:@__attributes_in_database, {author: 'john'}) + instance.instance_variable_set(:@__changed_model_attributes, {author: 'john'}) client.expects(:update).with do |payload| assert_equal 'foo', payload[:index] diff --git a/elasticsearch-model/test/unit/proxy_test.rb b/elasticsearch-model/test/unit/proxy_test.rb index e403b6141..a64b5b175 100644 --- a/elasticsearch-model/test/unit/proxy_test.rb +++ b/elasticsearch-model/test/unit/proxy_test.rb @@ -23,8 +23,6 @@ def self.before_save(&block) (@callbacks ||= {})[block.hash] = block end - def attributes_in_database; [:foo]; end - def changes_to_save {:foo => ['One', 'Two']} end @@ -43,10 +41,10 @@ def changes_to_save DummyProxyModelWithCallbacks.__send__ :include, Elasticsearch::Model::Proxy end - should "set the @__attributes_in_database variable before save" do + should "set the @__changed_model_attributes variable before save" do instance = ::DummyProxyModelWithCallbacks.new instance.__elasticsearch__.expects(:instance_variable_set).with do |name, value| - assert_equal :@__attributes_in_database, name + assert_equal :@__changed_model_attributes, name assert_equal({foo: 'Two'}, value) true end From 3b6f27d371c848d04488bf77a294fd1ff25929e4 Mon Sep 17 00:00:00 2001 From: Karel Minarik Date: Mon, 4 Dec 2017 18:03:07 +0100 Subject: [PATCH 13/20] Release 5.0.2 --- CHANGELOG.md | 6 ++++++ elasticsearch-model/lib/elasticsearch/model/version.rb | 2 +- .../lib/elasticsearch/persistence/version.rb | 2 +- elasticsearch-rails/lib/elasticsearch/rails/version.rb | 2 +- 4 files changed, 9 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bed561748..e1d3d07d2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,9 @@ +## 5.0.2 + +### ActiveModel + +* Fixed the deprecation warnings on Rails 5.1 due to calling the `changes` method + ## 5.0.1 ### ActiveModel diff --git a/elasticsearch-model/lib/elasticsearch/model/version.rb b/elasticsearch-model/lib/elasticsearch/model/version.rb index 14f022513..630fbbbd4 100644 --- a/elasticsearch-model/lib/elasticsearch/model/version.rb +++ b/elasticsearch-model/lib/elasticsearch/model/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Model - VERSION = "5.0.1" + VERSION = "5.0.2" end end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb index c8e1f2b1f..726db5fa1 100644 --- a/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Persistence - VERSION = "5.0.1" + VERSION = "5.0.2" end end diff --git a/elasticsearch-rails/lib/elasticsearch/rails/version.rb b/elasticsearch-rails/lib/elasticsearch/rails/version.rb index e2fc23ad4..da5948ed1 100644 --- a/elasticsearch-rails/lib/elasticsearch/rails/version.rb +++ b/elasticsearch-rails/lib/elasticsearch/rails/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Rails - VERSION = "5.0.1" + VERSION = "5.0.2" end end From 0239bd178e52c32dab73c1b586de43e31c25fba8 Mon Sep 17 00:00:00 2001 From: Emily Stolfo Date: Wed, 27 Jun 2018 12:32:59 +0200 Subject: [PATCH 14/20] Update gitignore --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 6076dc96c..43ddc2e30 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ .DS_Store *.log tmp/ +.idea/* .yardoc/ _yardoc/ From 8d79b98fd50b8500542c59b4840a86d8127c8e66 Mon Sep 17 00:00:00 2001 From: Emily S Date: Thu, 28 Jun 2018 16:32:49 +0200 Subject: [PATCH 15/20] [CI] Update Travis matrix for 5.x branch (#806) * [CI] Update Travis matrix * [MODEL] Mock the #to_ary method in multimodel test for Ruby 2.2 --- .travis.yml | 19 ++++++++++++++++--- elasticsearch-model/Rakefile | 6 +++--- .../test/unit/multimodel_test.rb | 4 ++-- 3 files changed, 21 insertions(+), 8 deletions(-) diff --git a/.travis.yml b/.travis.yml index 07aca469c..48f91d596 100644 --- a/.travis.yml +++ b/.travis.yml @@ -19,15 +19,28 @@ branches: matrix: include: - - rvm: 2.2.6 + include: + - rvm: 2.2 + jdk: oraclejdk8 + env: TEST_SUITE=unit + + - rvm: 2.3 jdk: oraclejdk8 env: TEST_SUITE=unit - - rvm: 2.3.3 + - rvm: 2.4 jdk: oraclejdk8 env: TEST_SUITE=unit - - rvm: 2.3.3 + - rvm: 2.5 + jdk: oraclejdk8 + env: TEST_SUITE=unit + +# - rvm: jruby-9.1 +# jdk: oraclejdk8 +# env: TEST_SUITE=unit + + - rvm: 2.5 jdk: oraclejdk8 env: TEST_SUITE=integration QUIET=y SERVER=start TEST_CLUSTER_LOGS=/tmp/log TEST_CLUSTER_COMMAND=/tmp/elasticsearch-5.3.0/bin/elasticsearch diff --git a/elasticsearch-model/Rakefile b/elasticsearch-model/Rakefile index 1efad46da..f80c46b23 100644 --- a/elasticsearch-model/Rakefile +++ b/elasticsearch-model/Rakefile @@ -38,10 +38,10 @@ namespace :test do sh "BUNDLE_GEMFILE='#{File.expand_path('../gemfiles/5.0.gemfile', __FILE__)}' bundle exec rake test:run_unit" end - desc "Run integration tests against ActiveModel 3, 4 and 5" + desc "Run integration tests against latest stable ActiveModel (5)" task :integration do - sh "BUNDLE_GEMFILE='#{File.expand_path('../gemfiles/3.0.gemfile', __FILE__)}' bundle exec rake test:run_integration" - sh "BUNDLE_GEMFILE='#{File.expand_path('../gemfiles/4.0.gemfile', __FILE__)}' bundle exec rake test:run_integration" + #sh "BUNDLE_GEMFILE='#{File.expand_path('../gemfiles/3.0.gemfile', __FILE__)}' bundle exec rake test:run_integration" + #sh "BUNDLE_GEMFILE='#{File.expand_path('../gemfiles/4.0.gemfile', __FILE__)}' bundle exec rake test:run_integration" sh "BUNDLE_GEMFILE='#{File.expand_path('../gemfiles/5.0.gemfile', __FILE__)}' bundle exec rake test:run_integration" end diff --git a/elasticsearch-model/test/unit/multimodel_test.rb b/elasticsearch-model/test/unit/multimodel_test.rb index 89e88f7a1..daf9f4043 100644 --- a/elasticsearch-model/test/unit/multimodel_test.rb +++ b/elasticsearch-model/test/unit/multimodel_test.rb @@ -4,8 +4,8 @@ class Elasticsearch::Model::MultimodelTest < Test::Unit::TestCase context "Multimodel class" do setup do - title = stub('Foo', index_name: 'foo_index', document_type: 'foo') - series = stub('Bar', index_name: 'bar_index', document_type: 'bar') + title = stub('Foo', index_name: 'foo_index', document_type: 'foo', to_ary: nil) + series = stub('Bar', index_name: 'bar_index', document_type: 'bar', to_ary: nil) @multimodel = Elasticsearch::Model::Multimodel.new(title, series) end From 931830cb73e41aa270c93463fb923893b41c57e6 Mon Sep 17 00:00:00 2001 From: Emily S Date: Fri, 29 Jun 2018 11:37:30 +0200 Subject: [PATCH 16/20] [CI] Update references to MongoDB Ruby driver (#807) --- .../test/integration/mongoid_basic_test.rb | 7 +++---- .../test/integration/multiple_models_test.rb | 6 +++--- elasticsearch-model/test/test_helper.rb | 9 ++++----- 3 files changed, 10 insertions(+), 12 deletions(-) diff --git a/elasticsearch-model/test/integration/mongoid_basic_test.rb b/elasticsearch-model/test/integration/mongoid_basic_test.rb index d46a75d05..c8b44dda0 100644 --- a/elasticsearch-model/test/integration/mongoid_basic_test.rb +++ b/elasticsearch-model/test/integration/mongoid_basic_test.rb @@ -1,9 +1,8 @@ require 'test_helper' +MongoDB.setup! -Mongo.setup! - -if Mongo.available? - Mongo.connect_to 'mongoid_articles' +if MongoDB.available? + MongoDB.connect_to 'mongoid_articles' module Elasticsearch module Model diff --git a/elasticsearch-model/test/integration/multiple_models_test.rb b/elasticsearch-model/test/integration/multiple_models_test.rb index 7d3a62705..02022b60f 100644 --- a/elasticsearch-model/test/integration/multiple_models_test.rb +++ b/elasticsearch-model/test/integration/multiple_models_test.rb @@ -6,7 +6,7 @@ ::ActiveRecord::Base.raise_in_transactional_callbacks = true if ::ActiveRecord::Base.respond_to?(:raise_in_transactional_callbacks) && ::ActiveRecord::VERSION::MAJOR.to_s < '5' -Mongo.setup! +MongoDB.setup! module Elasticsearch module Model @@ -115,8 +115,8 @@ class ::Series < ActiveRecord::Base assert_equal 0, response.page(3).per(3).results.size end - if Mongo.available? - Mongo.connect_to 'mongoid_collections' + if MongoDB.available? + MongoDB.connect_to 'mongoid_collections' context "Across mongoid models" do setup do diff --git a/elasticsearch-model/test/test_helper.rb b/elasticsearch-model/test/test_helper.rb index ff3a6d935..7471e475d 100644 --- a/elasticsearch-model/test/test_helper.rb +++ b/elasticsearch-model/test/test_helper.rb @@ -62,14 +62,13 @@ def setup end end -class Mongo +class MongoDB def self.setup! begin require 'mongoid' - session = Moped::Connection.new("localhost", 27017, 0.5) - session.connect + Mongo::Client.new(["localhost:27017"]) ENV['MONGODB_AVAILABLE'] = 'yes' - rescue LoadError, Moped::Errors::ConnectionFailure => e + rescue LoadError, Mongo::Error => e $stderr.puts "MongoDB not installed or running: #{e}" end end @@ -86,7 +85,7 @@ def self.connect_to(source) logger.level = ::Logger::DEBUG Mongoid.logger = logger unless ENV['QUIET'] - Moped.logger = logger unless ENV['QUIET'] + Mongo::Logger.logger = logger unless ENV['QUIET'] Mongoid.connect_to source end From 10028bb06c0bf91f33b4792feb3cd65af04bf702 Mon Sep 17 00:00:00 2001 From: Emily S Date: Wed, 4 Jul 2018 10:00:01 +0200 Subject: [PATCH 17/20] [STORE] Warn that the ActiveRecord persistence pattern is deprecated (#809) --- elasticsearch-persistence/README.md | 3 +++ .../lib/elasticsearch/persistence/model.rb | 18 ++++++++++++++++++ .../test/integration/model/model_basic_test.rb | 5 +++++ 3 files changed, 26 insertions(+) diff --git a/elasticsearch-persistence/README.md b/elasticsearch-persistence/README.md index 4c6dead10..a5eb65ade 100644 --- a/elasticsearch-persistence/README.md +++ b/elasticsearch-persistence/README.md @@ -445,6 +445,9 @@ and demonstrates a rich set of features: ### The ActiveRecord Pattern +Please note that this pattern is deprecated and will be removed in version 6.0. +The [Repository Pattern](#the-repository-pattern) is recommended instead. + The `Elasticsearch::Persistence::Model` module provides an implementation of the active record [pattern](http://www.martinfowler.com/eaaCatalog/activeRecord.html), with a familiar interface for using Elasticsearch as a persistence layer in diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/model.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/model.rb index ce948d4f3..7a51aafe3 100644 --- a/elasticsearch-persistence/lib/elasticsearch/persistence/model.rb +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/model.rb @@ -66,6 +66,22 @@ def gateway(&block) @gateway end + DEPRECATION_WARNING = "This (ActiveRecord) persistence pattern is deprecated. It will be removed in " + + "version 6.0 in favor of the Repository pattern. Please see the ReadMe for " + + "details on the alternative pattern.\n" + + "https://github.com/elastic/elasticsearch-rails/tree/master/elasticsearch-persistence" + + "#the-repository-pattern".freeze + + # Warn that this ActiveRecord persistence pattern is deprecated. + # + def raise_deprecation_warning! + if STDERR.tty? + Kernel.warn("\e[31;1m#{DEPRECATION_WARNING}\e[0m") + else + Kernel.warn(DEPRECATION_WARNING) + end + end + # Delegate methods to repository # delegate :settings, @@ -126,6 +142,8 @@ def deserialize(document) attribute :updated_at, Time, default: lambda { |o,a| Time.now.utc } attr_reader :hit + + raise_deprecation_warning! end end diff --git a/elasticsearch-persistence/test/integration/model/model_basic_test.rb b/elasticsearch-persistence/test/integration/model/model_basic_test.rb index 33555c980..e6a33515b 100644 --- a/elasticsearch-persistence/test/integration/model/model_basic_test.rb +++ b/elasticsearch-persistence/test/integration/model/model_basic_test.rb @@ -33,6 +33,11 @@ class ::Person Person.create_index! force: true end + should "warn that the ActiveRecord persistence pattern is deprecated" do + Kernel.expects(:warn).at_least_once + class ShouldWarn; include Elasticsearch::Persistence::Model; end + end + should "save the object with custom ID" do person = Person.new id: 1, name: 'Number One' person.save From 0b000bdf5f24f387f56a16b5b4fd721abd0706ca Mon Sep 17 00:00:00 2001 From: Emily Stolfo Date: Wed, 4 Jul 2018 14:56:44 +0200 Subject: [PATCH 18/20] Release 5.1.0 --- CHANGELOG.md | 6 ++++++ elasticsearch-model/lib/elasticsearch/model/version.rb | 2 +- .../lib/elasticsearch/persistence/version.rb | 2 +- elasticsearch-rails/lib/elasticsearch/rails/version.rb | 2 +- 4 files changed, 9 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e1d3d07d2..fa2b7dc70 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,9 @@ +## 5.1.0 + +### Persistence + +* Warn that the ActiveRecord persistence pattern is deprecated (#809) + ## 5.0.2 ### ActiveModel diff --git a/elasticsearch-model/lib/elasticsearch/model/version.rb b/elasticsearch-model/lib/elasticsearch/model/version.rb index 630fbbbd4..7b14e9491 100644 --- a/elasticsearch-model/lib/elasticsearch/model/version.rb +++ b/elasticsearch-model/lib/elasticsearch/model/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Model - VERSION = "5.0.2" + VERSION = "5.1.0" end end diff --git a/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb index 726db5fa1..41830a43e 100644 --- a/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb +++ b/elasticsearch-persistence/lib/elasticsearch/persistence/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Persistence - VERSION = "5.0.2" + VERSION = "5.1.0" end end diff --git a/elasticsearch-rails/lib/elasticsearch/rails/version.rb b/elasticsearch-rails/lib/elasticsearch/rails/version.rb index da5948ed1..5aa19c645 100644 --- a/elasticsearch-rails/lib/elasticsearch/rails/version.rb +++ b/elasticsearch-rails/lib/elasticsearch/rails/version.rb @@ -1,5 +1,5 @@ module Elasticsearch module Rails - VERSION = "5.0.2" + VERSION = "5.1.0" end end From 212b37a1e927015f2a095e0f841c897d377ad1c7 Mon Sep 17 00:00:00 2001 From: Emily Stolfo Date: Wed, 4 Jul 2018 15:09:02 +0200 Subject: [PATCH 19/20] Add yard as development dependency in Gemfile --- Gemfile | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Gemfile b/Gemfile index 4a7fe0473..a52a3c58c 100644 --- a/Gemfile +++ b/Gemfile @@ -8,3 +8,7 @@ gem 'elasticsearch-extensions' gem "pry" gem "ansi" gem "cane" + +group :development do + gem 'yard' +end \ No newline at end of file From bfdd2243f0de298547b2dc199e9a671cf72ccde6 Mon Sep 17 00:00:00 2001 From: Philip Yu Date: Fri, 17 Aug 2018 19:40:31 +0800 Subject: [PATCH 20/20] [MODEL] Support :scope, :query and :preprocess importing options for Mongoid (#786) * [MODEL] Support scope, query and preprocess importing options in Mongoid Adapter * test(mongoid-adapter): add testing for importing mongoid model with default scope --- .../elasticsearch/model/adapters/mongoid.rb | 13 +++- .../test/integration/mongoid_basic_test.rb | 64 +++++++++++++++++++ .../test/unit/adapter_mongoid_test.rb | 59 ++++++++++++++++- 3 files changed, 132 insertions(+), 4 deletions(-) diff --git a/elasticsearch-model/lib/elasticsearch/model/adapters/mongoid.rb b/elasticsearch-model/lib/elasticsearch/model/adapters/mongoid.rb index 5117dbf58..d4aff56f9 100644 --- a/elasticsearch-model/lib/elasticsearch/model/adapters/mongoid.rb +++ b/elasticsearch-model/lib/elasticsearch/model/adapters/mongoid.rb @@ -63,10 +63,17 @@ module Importing # @see https://github.com/karmi/retire/pull/724 # def __find_in_batches(options={}, &block) - options[:batch_size] ||= 1_000 + batch_size = options[:batch_size] || 1_000 + query = options[:query] + named_scope = options[:scope] + preprocess = options[:preprocess] + + scope = all + scope = scope.send(named_scope) if named_scope + scope = query.is_a?(Proc) ? scope.class_exec(&query) : scope.where(query) if query - all.no_timeout.each_slice(options[:batch_size]) do |items| - yield items + scope.no_timeout.each_slice(batch_size) do |items| + yield (preprocess ? self.__send__(preprocess, items) : items) end end diff --git a/elasticsearch-model/test/integration/mongoid_basic_test.rb b/elasticsearch-model/test/integration/mongoid_basic_test.rb index c8b44dda0..292c1f1a5 100644 --- a/elasticsearch-model/test/integration/mongoid_basic_test.rb +++ b/elasticsearch-model/test/integration/mongoid_basic_test.rb @@ -167,6 +167,70 @@ def as_indexed_json(options={}) assert response.results.any?, "Search has not returned results: #{response.to_a}" end end + + context "importing when the model has a default scope" do + class ::MongoidArticleWithDefaultScope + include Mongoid::Document + include Elasticsearch::Model + + default_scope -> { where(status: 'active') } + + field :id, type: String + field :title, type: String + field :status, type: String, default: 'active' + + attr_accessible :title if respond_to? :attr_accessible + attr_accessible :status if respond_to? :attr_accessible + + settings index: { number_of_shards: 1, number_of_replicas: 0 } do + mapping do + indexes :title, type: 'text', analyzer: 'snowball' + indexes :status, type: 'text' + indexes :created_at, type: 'date' + end + end + + def as_indexed_json(options={}) + as_json(except: [:id, :_id]) + end + end + + setup do + Elasticsearch::Model::Adapter.register \ + Elasticsearch::Model::Adapter::Mongoid, + lambda { |klass| !!defined?(::Mongoid::Document) && klass.respond_to?(:ancestors) && klass.ancestors.include?(::Mongoid::Document) } + + MongoidArticleWithDefaultScope.__elasticsearch__.create_index! force: true + + MongoidArticleWithDefaultScope.delete_all + + MongoidArticleWithDefaultScope.create! title: 'Test' + MongoidArticleWithDefaultScope.create! title: 'Testing Coding' + MongoidArticleWithDefaultScope.create! title: 'Coding' + MongoidArticleWithDefaultScope.create! title: 'Test legacy code', status: 'removed' + + MongoidArticleWithDefaultScope.__elasticsearch__.refresh_index! + MongoidArticleWithDefaultScope.__elasticsearch__.client.cluster.health wait_for_status: 'yellow' + end + + should "import only documents from the default scope" do + assert_equal 3, MongoidArticleWithDefaultScope.count + + assert_equal 0, MongoidArticleWithDefaultScope.import + + MongoidArticleWithDefaultScope.__elasticsearch__.refresh_index! + assert_equal 3, MongoidArticleWithDefaultScope.search('*').results.total + end + + should "import only documents from a specific query combined with the default scope" do + assert_equal 3, MongoidArticleWithDefaultScope.count + + assert_equal 0, MongoidArticleWithDefaultScope.import(query: -> { where(title: /^Test/) }) + + MongoidArticleWithDefaultScope.__elasticsearch__.refresh_index! + assert_equal 2, MongoidArticleWithDefaultScope.search('*').results.total + end + end end end diff --git a/elasticsearch-model/test/unit/adapter_mongoid_test.rb b/elasticsearch-model/test/unit/adapter_mongoid_test.rb index ca9b0d20b..0074df097 100644 --- a/elasticsearch-model/test/unit/adapter_mongoid_test.rb +++ b/elasticsearch-model/test/unit/adapter_mongoid_test.rb @@ -98,7 +98,64 @@ def ids assert_equal @transform.call(model), { index: { _id: "1", data: {} } } end end - end + should "limit the relation to a specific scope" do + relation = mock() + relation.stubs(:no_timeout).returns(relation) + relation.expects(:published).returns(relation) + relation.expects(:each_slice).returns([]) + DummyClassForMongoid.expects(:all).returns(relation) + + DummyClassForMongoid.__send__ :extend, Elasticsearch::Model::Adapter::Mongoid::Importing + DummyClassForMongoid.__find_in_batches(scope: :published) do; end + end + + context "when limit the relation with proc" do + setup do + @query = Proc.new { where(color: "red") } + end + should "query with a specific criteria" do + relation = mock() + relation.stubs(:no_timeout).returns(relation) + relation.expects(:class_exec).returns(relation) + relation.expects(:each_slice).returns([]) + DummyClassForMongoid.expects(:all).returns(relation) + + DummyClassForMongoid.__find_in_batches(query: @query) do; end + end + end + + context "when limit the relation with hash" do + setup do + @query = { color: "red" } + end + should "query with a specific criteria" do + relation = mock() + relation.stubs(:no_timeout).returns(relation) + relation.expects(:where).with(@query).returns(relation) + relation.expects(:each_slice).returns([]) + DummyClassForMongoid.expects(:all).returns(relation) + + DummyClassForMongoid.__find_in_batches(query: @query) do; end + end + end + + should "preprocess the batch if option provided" do + class << DummyClassForMongoid + # Updates/transforms the batch while fetching it from the database + # (eg. with information from an external system) + # + def update_batch(batch) + batch.collect { |b| b.to_s + '!' } + end + end + + DummyClassForMongoid.expects(:__find_in_batches).returns( [:a, :b] ) + + DummyClassForMongoid.__find_in_batches(preprocess: :update_batch) do |batch| + assert_same_elements ["a!", "b!"], batch + end + end + end end end