AI/Machine Learning 2022 Review and 2023 Outlook — A Practitioner’s Perspective

In the last day of 2022, it’s always a great opportunity to summarise what had happened and to look ahead to how AI will further evolve. Of course, what I mention below is not exhaustive and is more of my personal opinion. I intend not to spend a lot time writing this article because it reflects the big picture I have in mind, and also a snapshot of my view.

2022 has been an exciting year of AI, where we see some critical and influential achievements in AI, including but not limited to:

• release of some mega size datasets like LAION-5B, that is critical to the success of Stable Diffusion
• Stability AI releasing open-source Stable Diffusion models, which disrupts art & design industry, and multiple variants of fine-tuned models were developed upon it
• BLOOM, an open-source large language model whose development is boostrapped by HuggingFace, GENCI and IDRIS
• launch of OpenAI’s ChatGPT, which leads to countless use case in multiple domains, from writing a joke/ a letter/ a contract/ code/ book to asking for technical/financial/ life advice

From the perspective of an applied machine learning researcher and practitioner, 2022 has shown that:

From Large Language Model to AGI

• we are much closer to AGI (artificial general intelligence) than we thought (well there is still dispute but we can see large language model could pass Turing test in multiple settings. Simply the definition of AGI, and its testing worth a long article)
• way to AGI may be more about aligning large model behaviour with human behaviour as we see the difference betweenOpenAI’s GPT3.5 and ChatGPT
• scaling law in language model continues to hold true
• large language model no longer concerns about overfitting, as it memorises all knowledge possibly collected in the universe
• transformer can handle very long sequences, with more GPUs

Image Generation

• text is very important to control image generation, much better than predetermined classes
• diffusion model becomes the de facto image generation architecture

Open-source Collaboration

• open-source collaboration is a successful working model for AI

I am very excited about what will happen in 2023. Below are my predictions for 2023:

From Large Language Model to AGI

• larger language model (which does not need prediction, as we see GPT-4 is coming)
• effort to understand why large language model is so good at zero shot learning/ look for smaller model size without performance degradation
• continuous effort to align model behaviour with human behaviour via Reinforcement Learning
• giving large model’s access to physical world
• development of rigorous AI reasoning and causation test and dataset
• more multimodal attempt

AI Safety

• more advanced detection of unfair use of large model, as generative model becomes more advanced

Open-source Collaboration

• development of open-source ChatGPT-like system (which is already happening)

As ML practitioners, given the rise of large language model, there are a lot of opportunities ahead:

• have a productivity boost with ChatGPT-like system
• do more creative and deeper work that can’t be found in ChatGPT-like system, better understanding of model behaviour, more rigorous scientific process and less on coding
• produce specialised/ small version of ChatGPT-like system that can be deployed in private cloud
• leverage ChatGPT-like system as a human-machine interface and a workflow orchestrator
• building inputs for ChatGPT-like system
• participate into open-source collaboration
• be ready to finetune large model whose weight can’t be fit in 1 GPU

Again, all the above are my personal views and its not meant to be a scientific and complete review, as I do not spend more than two hours writing this.

Last but not least, wish everyone a happy new year!

Note 1 : Opinions expressed are solely my own and do not express the views or opinions of my employer.

Note 2: The article is not generated by ChatGPT nor curated by it.

Note 3: Machine learning and AI are used interchangeably in this article because the recent advancement in AI is mostly fuelled by machine learning.