GPT-3 : The start of general A.I
In my previous article, I have discussed about how A.I was originated and how it is becoming creative. If you liked it you’re going to be fascinated with this one. We’ll be diving into some very interesting stuff. In December of 2017 a group of researchers asked themselves will A.I be writing most of the computer code out there? And they estimated that this could happen by 2040 and the thing is we may be already seeing the start of it. We know that AutoML by Google is an A.I that can write computer code and allow developers with limited expertise to build ML models and train/test them.
But this next A.I GPT-3 or Generative Pre-trained Transformer 3 is a step above that in terms of diversity. It is a deep learning algorithm that produces human-like text. It is the third generation language prediction model created by San Francisco startup OpenAI which was co-founded by Elon Musk. This program is better than any prior program at producing text which could have been written by a human. The reason that this is such a breakthrough is that it may prove useful to many companies and has great potential in automating tasks. Just imagine an A.I that can write anything you feed it like poems from a particular poet and it will write a new one with the same rhythm and genre, it can write news articles like this one published by The Guardian.
Yes this news article was produced by an A.I, it can write computer code in any language it can read an article answer questions from the information in the article and even summarize that article for you. One more impressive thing is it can also generate pictures from text some of these things aren’t new by themselves but doing all of these things under the same algorithm is pretty impressive. The internet is buzzing with its release David Chalmers an Australian philosopher described GPT-3 as,
“One of the most interesting and important A.I systems ever produced.”
GPT-3 Beta Test Application Demos :
Now let me show you some examples of beta testers on twitter putting the uses of this new algorithm.
- Here it is writing some java code just when given a text description
2. How about making a mock-up website by just copying and pasting a url with a description.
3. In this example after receiving the input of a complicated explanation of bitcoin the A.I summarizes it and explains it in simple terms.
4. How about building a machine learning model just by describing the data set and required output, no coding is required.
5. What about a Turing test, A.I usually struggles at common sense but GPT-3 easily manages these examples. You can see that it possesses great abstraction behind its implementation.
6. And the cherry on the cake is, it can generate faces from a text description.
I’m honestly impressed at how general this ai seems.
How does it work?
GPT-3 is a massive neural network that has the capacity of 175 billion
machine learning parameters. it was trained on hundreds of billions of words including books, Wikipedia and the general web which includes coding. It’s training data is like one time operation and doesn’t require further training for specific language tasks. It is smart enough to apply what it’s learned to many other things without human supervision & it is 10 times bigger than the previous largest language learning model which was made by Microsoft in February 2020. As a general rule the larger the number of parameters, the more accurate the A.I will be. For text GPT-3 calculates how likely one word is to appear in a text given the other words in the text this technique is known as the “Conditional probability of words“. For example in the sentence,
It’s very hot here, I want to turn on the ________.
The blank can be filled with any word, there are countless possibilities but from what the A.I has learned is, it thinks that the probabilities of AC/Air Conditioner being the next word of this sentence is higher than the word fridge/oven.
It’s very hot here, I want to turn on the AC.
The quality of the text generated by the A.I can be so high that it’s difficult to distinguish from that of a human. It has both benefits and risks associated with it.
Potential Risks :
OpenAI researchers and engineers warned of GPT-3 as a potential dangers and called for research to control them. They have analysed the harmful effects and consequences including misinformation, fake news articles, spam, phishing and fraudulent academic essay writing and so forth. If you’re in the school of languages just give the assignment to the A.I to read and ask it to write an essay for you and then modify if needed. There’s even been a case of a university student who used the A.I to automatically write blog entries under a fake name and that blog garnered some interest and some people even subscribed to it believing it was a human who wrote those articles. Some of the articles were so good that it reached the top spot in Hacker News.
Shortcomings :
There is still a really long way to go. This A.I doesn’t understand context, it just understands the rules of language and has mastered them. But it truly doesn’t know what it’s saying or has really understood knowledge about the world. MIT Technology Review did a great work on its shortcomings. Here are some examples of it making mistakes.
Context : You poured yourself a glass of cranberry juice but then you absentmindedly poured about a teaspoon of grape juice into it. It looks okay so you try sniffing it, but you have a bad cold, so you can’t smell anything. You are very thirsty. So :
A.I Response : You drink it, You are now dead.
In this example the A.I assumed that grape juice was a poison and that was because of the structure of the preceding sentences. Here’s another example,
Context : You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive french couture; it was a birthday present from Isabel. You decide that you should wear :
A.I Response : the bathing suit to court. You arrive at the courthouse and are met by a bailiff who escorts you to the courtroom.
The phrase however your bathing suit is clean though the AI doesn’t really understand that a bathing suit isn’t a proper suit and of course no lawyer would wear a bathing suit to a court. One last example is,
Context : You are having a small dinner party. You want to serve dinner in the living room the dining room. Table is wider than the doorway, so to get it into the living room, you will have to:
A.I Response : Remove the door. You have a table saw, so you cut the door in half and remove the top half.
This really doesn’t make much sense at all. Logically no one would do this moving the table to the side and removing the legs if possible would make more sense. One more example of a mistake is here you can see it being mislead to give wrong answers that defy common sense.
In conclusion GPT-3 can write sentences well enough and fool us humans but it’s no deeper that it is mindless and doesn’t demonstrate much broader understanding. But again on the flip side this is the very worst that this technology will ever be. In the world of A.I we could be at the 1950s of computing where things are rudimentary and fail often but in three years we may be at the equivalent of 1980s. I think completing a year in the A.I time is equal to multiple decades of regular technology progress, so we’ll have to wait and see. While some news sources might say that this is the end of society and we’re all doomed, I believe both machines and humans can go hand in hand.
So what do you guys think? I’ll be interested to know your thoughts on it. I hope I’ve given you a brief idea about the start of general A.I - GPT-3, it’s applications and short comings. Give it a 💚, if you like this post for extra motivation. I am always open to your suggestions and queries.
