You [Gerald Bauer¹] have been permanently banned [for life] from participating in r/ruby (because of your writing off / outside of r/ruby). I do not see your participation adding anything to this [ruby] community.

-- Richard Schneeman (r/ruby mod and fanatic illiberal ultra leftie on a cancel culture mission)

¹: I know. Who cares? Who is this Gerald Bauer anyway. A random nobody for sure. It just happens that I am the admin among other things of Planet Ruby.

Case Studies of Code of Conduct "Cancel Culture" Out-Of-Control Power Abuse - Ruby - A Call for Tolerance On Ruby-Talk Results In Ban On Reddit Ruby

Update (August, 2022) - A Call for More Tolerance And Call For No-Ban Policy Results In Ban On Ruby-Talk (With No Reason Given) 

>  I just banned gerald.bauer@gmail.com.
>
>  -- SHIBATA Hiroshi
>
>> THANK YOU
>> 
>>  -- Ryan Davis
>>
>>
>> My full support to moderators.
>>
>> -- Xavier Noria
>> 
>> My full support to moderators.
>>
>>  -- Carlo E. Prelz
>>
>>  That's fun.
>>
>>  -- Alice

Read the full story »

« 25 Days of Ruby Gems - Ruby Advent Calendar 2020, December 1st - December 25th

Day 2 - elastic-enterprise-search Gem - Workplace Search, App Search, and Site Search All Together

Written by picandocodigo Fernando Briano

Software engineer at Elastic maintaing among other gems elasticsearch-api, elasticsearch-transport, elasticsearch, elasticsearch-model, elasticsearch-rails, elasticsearch-dsl, elasticsearch-persistence, elasticsearch-extensions, elasticsearch-xpack, elastic-app-search, elastic-enterprise-search, elastic-workplace-search. Blogs at Picando Código. Plays Nintendo. Listens to Punk Rock. Reads books. Drinks beer.

This gem is the official Elastic Enterprise Search client and it’s currently out as a beta release. Enterprise Search is an Elastic solution for search in the enterprise. It’s powered by the Elastic Stack and Elasticsearch, so you get all the great things you already know in one package for your team. It’s the umbrella solution which contains Workplace Search, App Search and Site Search. The client focuses on App Search and Workplace Search.

Workplace Search is a unified search solution for teams. One of the interesting things about Workplace Search to me is at previous jobs I wished I had a product like this. We had different documents and manuals on so many different services: wikis, Jira, email, Dropbox, Basecamp. Whenever you needed to find a specific document you had to find out where to look for it, and then search. Workplace gives you a single search experience across all your content. There are already built-in integrations for Google Drive, Sharepoint, GitHub and other popular services. But you can also add Custom Document Sources if you’re using a system which doesn’t have an integration yet and start indexing documents.

App Search is a ready to use search solution with built in analytics and user friendly relevance tuning. You can use it to add this search experience to an app, web, etc. In App Search you create engines, information repositories that specialize in search. It’s really easy to index JSON documents and make them accessible from search right away, you don’t need to specify a schema, since it’s created automatically for you.

The Enterprise Search gem

Based on our experience maintaining the Elasticsearch clients, one of the ideas we had during the development of this client was to reuse the Elasticsearch client’s HTTP transport layer. These clients are pretty well tested and they’ve been designed in a modular way. So we opened a branch to include the dependency on elasticsearch-transport, removed all the http connection code from the new clients and tried it out. It was rather easy, as you can see in the following code, it’s optimized for use with Elasticsearch but it’s an HTTP library:

require 'elasticsearch-transport'

client = Elasticsearch::Transport::Client.new(
  host: 'http://jsonplaceholder.typicode.com:80'
)

client.perform_request('get', 'posts/42').body

#=> {"userId"=>5, "id"=>42, "title"=>"commodi ullam sint et excepturi error explicabo praesentium voluptas",
#  "body"=>"odio fugit voluptatum ducimus earum autem est incidunt voluptatem\n
#           odit reiciendis aliquam sunt sequi nulla dolorem\n
#           non facere repellendus voluptates quia\nratione harum vitae ut"}

The Enterprise Search client was built with the HTTP layer from the Elasticsearch client and the APIs from each service in Enterprise Search. The code for each of the APIs is generated from an OpenAPI JSON Spec which we’ve worked on together with the Enterprise Search, Workplace Search and App Search teams. So when the API for any of these changes in a new version, we just need to update the Open API spec and regenerate the code.

With these clients your app can integrate seamlessly with Elastic Enterprise Search. You can check out the code on GitHub or install from RubyGems:

$ gem install elastic-enterprise-search --pre

The source code includes a rake task for running the Enterprise Search stack via Docker: rake stack[version], but you can also set up an instance in Elastic Cloud. Once you have an instance running, either via Docker or Cloud, you can try out the client:

require 'elastic-enterprise-search'

# These are the default credentials if you're running the stack rake task:
http_auth = { user: 'elastic', password: 'changeme' }
host = 'localhost:3002'

client = Elastic::EnterpriseSearch::Client.new(log: true, host: host, http_auth: http_auth)
response = client.health

The response object is an instance of Elasticsearch::Transport::Transport::Response from elasticsearch-transport. You can call headers or status on it, but you’ll usually use response.body to see the JSON response from the server. The EnterpriseSearch::Client API gives you some useful endpoints to check the status of your cluster and managing read-only mode. You can read more here.

For Workplace Search and App Search, you can instantiate a new client or share the transport layer with an Enterprise Search client you’ve already created. This means they’ll share the same host, but both Workplace Search and App Search have their own authentication method.

To start using the client, we need to open our Enterprise Search web UI on our browser and select Workplace Search from the initial screen. Once there, we’ll see a screen that says “Shared content sources” and a button to add sources. We can also add sources from the “Sources” item on the menu on the left. On the “Add a shared content source” screen, we need to select “Custom API Source”, give it a name and press “Create Custom API Source”:

Custom API Source

Once the source is created we get an Access Token and a Key.

Ruby Client Created

Given the client instantiated on the previous example, we can now start interacting with Workplace Search:

access_token = '<PASTE Access Token FROM WORKPLACE SEARCH>'
content_source_key = '<PASTE Key FROM WORKPLACE SEARCH>'
workplace_search = client.workplace_search(http_auth: access_token)

We’re going to use the content_source_key when calling the API to indicate which source we’re using, in this case the one we just created. Let’s try to index some documents:

documents = [
  { title: '20,000 Leagues Under the Sea', year: 1916, director: 'Stuart Paton' },
  { title: 'Dr. Jekyll and Mr. Hyde', year: 1920, director: 'John S.Robertson' },
  { title: 'Frankenstein', year: 1910, director: 'J. Searle Dawley' },
  { title: 'Metropolis', year: 1927, director: 'Fritz Lang' },
  { title: 'Nanook of the North', year: 1922, director: 'Robert J. Flaherty' },
  { title: 'Night of the Living Dead', year: 1968, director: 'George A. Romero' },
  { title: 'Plan 9 from Outer Space', year: 1959, director: 'Ed Wood' },
  { title: 'Popeye the Sailor Meets Ali Baba\'s Forty Thieves', year: 1937, director: 'Dave Fleischer' },
  { title: 'The Immigrant', year: 1917, director: 'Charles Chaplin' }
]

response = workplace_search.index_documents(content_source_key, body: documents)

The response body should show us the id’s that Workplace Search generated automatically for our documents and error arrays for each of them, which will be hopefully empty if nothing went wrong. Now that we’ve indexed some documents, we can visit {host}/ws/search (http://localhost:3002/ws/search if you’re running the repository’s docker container) and see that our documents have been indexed and we can search through them.

The client provides more Workplace Search functionalities you can read about here.

To use App Search we need to go back to the Enterprise Search Home in our browser and launch App Search. You can skip the onboarding, and you will be presented with the “Create Engine” screen. We’re going to create an engine from the client, but we need to access the Credentials page from the left menu. You can create a new API Key for the client, or for this example use the provided private-key which has read/write access. Copy the value for this key and paste it in your Ruby code. As with Workplace Search, we can create a new client or share the transport layer with an Enterprise Search client we already created:

api_key = '<PASTE Api Key FROM UI>'
app_search = client.app_search(http_auth: api_key)

engine_name = 'books'
response = app_search.create_engine(name: engine_name)
app_search.list_engines

Now that we’ve created an engine, let’s index some documents again:

documents = [
  { title: 'Beowulf: An Anglo-Saxon Epic Poem', author: 'J. Lesslie Hall' },
  { title: 'Frankenstein; Or, The Modern Prometheus', author: 'Mary Wollstonecraft Shelley' },
  { title: 'Jungle Tales', author: 'Horacio Quiroga' },
  { title: 'Lenguas de diamante', author: 'Juana de Ibarbourou' },
  { title: 'Leviathan', author: 'Thomas Hobbes' },
  { title: 'Metamorphosis', author: 'Franz Kafka' },
  { title: 'Moby Dick; Or, The Whale', author: 'Herman Melville' },
  { title: 'Pride and Prejudice', author: 'Jane Austen' },
  { title: 'Shetland Folk Tales', author: 'Lawrence Tulloch' },
  { title: 'Siddhartha', author: 'Hermann Hesse' },
  { title: 'The Jungle Book', author: 'Rudyard Kipling' },
  { title: 'The Masque of the Red Death', author: 'Edgar Allan Poe' },
  { title: 'The Republic', author: 'Plato' },
  { title: 'The Strange Case of Dr. Jekyll and Mr. Hyde', author: 'Robert Louis Stevenson' },
  { title: 'The War of the Worlds', author: 'H. G. Wells' }
]

app_search.index_documents(engine_name, body: documents)

And we can also search:

app_search.search(engine_name, body: { query: 'Jungle' })
app_search.multi_search(engine_name, queries: [{ query: 'Jungle'}, {query: 'Herman' }])

You can also take advantage of the App Search UI to interact with the documents you’ve just indexed, look at the API logs or create a reference UI for the engine you created. The client provides more App Search functionalities you can read about here.

Conclusion

The gem is still in beta but it’s under active development. Take it for a spin and check out the repo to report any issues or contribute. And if you want to learn more about Enterprise Search, checkout these free Quick Start guides.

There are currently API clients for App Search and Workplace Search packaged separately as Ruby gems. These clients still work but will be deprecated in favor of the new API client once the new client reaches general availability in Elastic Enterprise Search in a future release. Migrating to the new API client likely won’t require much effort due to similarities between the APIs.

Find Out More

References

Built with Ruby (running Jekyll) on 2023-01-25 18:05:39 +0000 in 0.371 seconds.
Hosted on GitHub Pages. </> Source on GitHub. (0) Dedicated to the public domain.