Transformer

not this transformer

A friend (MC) who read my previous post recommended me to check out this software at https://transformer.huggingface.co/ and I must say it is pretty cool. I wrote about something like this last year when I shared about another transformer. In fact, the site I briefly wrote about was called https://talktotransformer.com/.

So what are 'Transformers'? No, not the ones from the cartoon/movie. Giuliano Giacaglia wrote about it in this article 'How Transformers work?: The Neural Network used by Open AI and DeepMind"

"Transformers are a type of neural network architecture that have been gaining popularity. Transformers were recently used by OpenAI in their language models, and also used recently by DeepMind for AlphaStar — their program to defeat a top professional Starcraft player.

Transformers were developed to solve the problem of sequence transduction, or neural machine translation. That means any task that transforms an input sequence to an output sequence. This includes speech recognition, text-to-speech transformation, etc.." 

Jay Alammar's visualisation of a Neural Machine Translation Network

(Read more about transformers and more from the site Towards Data Science)

Now that we roughly know what Transformers are let's see how it' works. I have embedded the site my friend suggested earlier. Just start click on one of the 'Start Writing' button and check out the magic.

Look how the Transformer continued based on my prompts.

I tried to type out 2 lines and pressed tab to get suggested sentences to continue my story using the distilled GPT-2. Students could reallly have fun seeing how a 'machine' could complete their story but I am not sure if the final product would be good enough as there's a chance it would go out of point.

A counter-checking tool in case the Transformers are abused to create fake news or spams

You can have a try and see how the GLTR analyses text to determine if it was written by a human or not.

See the analysis provided from GLTR

It would be interesting to see this being used in Humanities subjects as a complement for example to build critical thinking on the reliability of texts. However, the system is not perfect and this site only offers a demo. When I tried it with the text from earlier, the analysis hanged the site.

I hope you will try it out for yourself with your students. Do update me if there's any interesting findings or stories that you get out of the above Transformers.

A colleague also shared an interesting AI tool that could detect Sarcasm as part of a presentation about whether computers can or should know our emotions. The tool called Crystalace is apparently a creation of Singapore's A*STAR.