In this article, we would like to introduce Mr. Yusuke Iwasawa.
Mr Iwasawa joined the Matsuo-Iwasawa Laboratory (hereinafter Matsuo Lab) during his doctoral program and has been conducting research since then in the roles of specially appointed researcher, assistant professor, and lecturer.
In this article, we asked Mr. Iwasawa, who has been at the Matsuo Lab for 9 years, about his research and the lab’s appeal.
(Updates in January 2024) Titles are as of the time of the interview in December 2022.
He is currently an associate professor at the School of Engineering at the University of Tokyo.
Promoting a wide range of research, from individual to large-scale, in collaboration with the engineering team.
ーWhat kind of work and research do you do at Matsuo Lab?
Regarding work content, I don’t think it’s different from what you might imagine as work in normal academic work. Our main focus is on research, but we also provide education and guidance to students. However, there are various types of research; some are conducted on an individual motivation basis, others are top-down projects, and the rest are conducted in collaboration with the engineering teams.
My recent research has focused on efficiently utilizing large-scale models in new tasks. In the machine learning research domain, this is known as transfer learning, meta-learning, and out-of-distribution generalization. In particular, in light of recent trends showing the effectiveness of very large-scale models in various domains, we are researching techniques (i.e., Test-Time Adaptation and Prompting) for efficiently adapting even larger-scale models to unknown environments and tasks.
In addition to individual or small-scale research, some research is conducted by teams while collaborating with the engineering teams.
One factor that makes organizations such as GAFA (Google, Amazon, Facebook, Apple) leaders in machine learning research is that, along with a good group of researchers, they are well-equipped to conduct large-scale experiments. Collecting data as a preliminary step in the research process and other aspects that cannot be completed by simply examining algorithm ideas are necessary to promote high-impact research.
Of course, the scale of our research is smaller than that of a large corporation. However, using a similar scheme, we can still promote large-scale research that would require much more work for individuals to achieve. I feel that we are blessed to be able to freely conduct and lead such research.
(Reference)
ー What kind of activities do you do outside of research?
One of our long-running non-research activities is the Deep Learning reading group, conducted weekly. This study group, which has been ongoing since 2015, meets every Friday to read the latest papers on Deep Learning on a rotation basis. Currently, people from various backgrounds, mainly graduates of Matsuo Lab lectures, are participating in this project.
The translation of a book written by Ian Goodfellow and other well-known people in the deep learning community was also started due to the reading group activities. Prof. Matsuo, who participated in the reading group, said, “Why don’t you get the copyright and translate the book?”. At that time, I was still a doctoral student, and I remember how shocked I was at the progress of the translation. I was involved in both direction translation work and its overall supervision. Although it was challenging, I think it was a valuable experience.
In addition, I have been involved in lecturing and organizing courses on deep learning. I also frequently provide guidance to students. In April, I became a lecturer at the University of Tokyo and officially started accepting students.
ー What was your motivation for becoming a lecturer?
The main reasons were that I wanted to learn more about the university and to expand the number of undergraduate students we could officially accept at the laboratory.
Thankfully, many students apply to be assigned to Matsuo Lab every year, but only a few are accepted due to the limited number of positions available.
Of course, affiliation is not everything, but we wanted to create as many opportunities as possible for motivated students to conduct research in deep learning and intelligence. I decided to take this position with the hope that I could accept more aspiring students as a member of Matsuo Lab. `
Research on machine learning is an important topic with applications in various areas
ー How did you join Matsuo Lab?
The starting point of my research was the application of machine learning to support people with disabilities at Sophia University. The research was more of an application-oriented research area. The goal was to make a map to visualize where in the city dangerous behaviors (such as falls and collisions) and near-misses that lead to such hazardous behaviors were occurring. The data was collected using sensors mounted on devices such as iPhones to collect the behavior of wheelchair users and the visually impaired and analyze using machine learning.
Through this research, I was exposed to “machine learning” for the first time. I was the only one doing machine learning in my laboratory back then. It was around 2011, so neural networks and deep learning were not so well-known.
In such a situation, I attended a conference where my former academic advisor introduced me to Prof. Matsuo. I wanted to study this technology more, so I started my research at Matsuo Lab from the doctoral course.
ー So you were not researching the current theme from the beginning?
I wonder if I am allowed to say this as someone supervising the research team. Still, I did not have a strong interest in “creating intelligence” from the beginning (At this point in time, I believe this is a unique opportunity to work on the very challenging issue of the principles of intelligence while experiencing the technological advancement of deep learning).
I was originally interested in application and machine learning as a problem-solving tool because I had been researching support for people with disabilities. However, as I continued my research, the core problem of how to adapt the learned models to new situations (new users and other environments) remained. I also realized that this problem was not unique to the task I was working on at the time but was relevant to general applied machine learning.
Eventually, I realized that looking at the problem from an abstract perspective, no matter which angle I looked, I would end up with the same problem. This narrowed my awareness of the problem to the area of transfer learning, which is my current main research topic.
In an environment of respect for researchers, we promote basic research with a view to long-term return to society.
ー It’s been nine years since you joined Matsuo Lab, but why do you continue working there?
There are many reasons, why it is difficult to choose one, but one of them is that researchers are respected in terms of work content, treatment, and psychological aspects. I cannot say that I am not busy, but I have a great deal of time to devote to important areas such as research and education. For example, the education component, which requires considerable effort to operate, is carried out very efficiently thanks to the presence of staff dedicated to lectures.
Many people may think that Matsuo Lab is actively engaged in collaborative research and nurturing entrepreneurs, which are forms of applied research that can be useful to society in the short term. Of course, we emphasize whether our research will broadly benefit society. Still, I feel that we are an organization that understands that the usefulness of our research belongs to a long value chain. I think we focus more on diverse and fundamental research than we are known for.
On a more operational level, it is a great opportunity to regularly have discussions with Prof. Matsuo and see what he is thinking up close.
From watching Prof. Matsuo closely for many years, I have the impression that he sees things on a different scale. It is not uncommon for researchers to be stuck in a narrow perspective as research progresses, and the research tends to aim to solve a specific problem. If you do your research well, you can still produce papers, but I think it is difficult to produce impactful results.
In this respect, Prof. Matsuo’s comments offer a very high perspective, which often leads to a great expansion of my thinking (although it sometimes takes me a while to understand them, to be honest). I believe that this may be attractive to other researchers as well.
To broaden our narrow vision, we need to think from the first principle
ー What exactly do you mean by “high vision”?
I think it is to question assumptions that no one would question and to create a structure not confined to the research community.
For example, in machine learning, we formulate a problem and solve it as an optimization problem, but in the end, the most difficult part is the formulation. If a good formulation can be made, the problem can be solved. In this context, I think that most ordinary research is about “how to reduce the problem to a solvable problem setting” or, in other words, “how to find research within a solvable problem setting. Of course, such research is important, but it is limited in principle. The great thing is to return to the fundamental question, “Let’s create a way to solve that problem.”
Of course, it takes time to realize this kind of research. That is not to say that we are only talking in the abstract, but I have the impression that our thinking is multi-layered, going back and forth between the concrete and the abstract.
Prof. Matsuo’s high level of enthusiasm propagated and significantly changed his surroundings.
ーWhat kind of person is Prof. Matsuo to you, Mr. Iwasawa?
I often think that he is pretty educational.
I don’t know if “educational” is the right word, but he doesn’t instruct me in detail or ask me many questions. Most of the time, he asks me questions about things that I do not understand well or gives me suggestions from a different point of view than the one I am considering.
The style of education I had initially envisioned was more like teaching step by step. Still, after meeting Prof. Matsuo, I realized that there are other kinds of education.
I also have the impression that he is always seriously challenging himself.
For example, many people feel a sense of stagnation in Japan or who have something they want to do, but I think there are very few people who are taking action and conducting research. But when Prof. Matsuo says, “I want to do something,” he has already started.
If he wants to do something, he will think of a way to do it, and he will do it. It may seem obvious when you say it, but I think it is amazing. This kind of serious approach to things has a great educational effect.
ー What would you like to do at Matsuo Lab in the future?
The mission of the Matsuo Lab’s basic research is “ to realize intelligence,” and I think it is important to move forward toward this major goal. Of course, it is a very big goal, and I do not think it will be easy, but now that deep learning research has advanced so much, I believe that the timing is right for us to have a large role to play in this challenge.
There are still some important research areas left to be done. Still, one major issue is to have a unified understanding of what we have learned so far in deep learning and technological progress and how to unify this understanding. Although there has been a lot of progress in learning sub-modules that have been reduced to an appropriate optimization problem through deep learning and increased data, I believe that there are still issues to be solved in terms of how to combine modules that already exist or what optimization problem to reduce them to in order to combine them. Intelligence is not simply a matter of creating modules but is an integrative process, so I think it is necessary to advance the theory and application of this process.
I think that acquiring data and continuously learning from new data are also important issues. After all, recent results in deep learning have shown that an increase in the scale of data, models, and computation causes a large increase in performance, but there isn’t a clear solution as to what kind of data should be collected. Technically, I think some research is being done on mechanisms to proactively acquire new data and continue learning from ever-changing data without avoiding collapse. Also, I think it is necessary to think about this from a business perspective, just as OpenAI continuously collects and improves data by opening its GPT API to the public.
ー Lastly, please give a message to your future colleagues.
The appeal of the Matsuo Lab is how it continues to change. In fact, in the nine years that I have been in the lab, it has undergone various changes, including a full-fledged shift from web-based research to deep learning research, the launch of several endowed courses, the opening of the School of Metaverse Engineering, and has grown as an organization. Making changes is challenging, but it also creates opportunities. I also believe we are now in a situation where the foundations are established to promote research with a long value chain. I would like to invite people willing to take on difficult problems, whether in research or business, to join us.
(Profile)
Yusuke Iwasawa / Lecturer at Matsuo Laboratory of the University of Tokyo
Career
- March, 2014: Graduated from Graduate School of Science and Engineering, Sophia University with a major in informatics
- March, 2017: Graduated from Graduate School of Engineering, The University of Tokyo
- April, 2017 ~ June, 2018: Specially appointed researcher Fellow, Graduate School of Engineering, University of Tokyo
- July, 2018 ~ November, 2020: Specially Appointed Assistant Professor
- December, 2020 ~ March, 2022: Specially Appointed Lecturer
- April, 2022 ~ December, 2023: Lecturer (as of the time of the interview)
- January, 2024 ~ present: Associated Professor (updated in May, 2024)
Area of specialty
- Transfer learning, deep representation learning, and deep learning applications
Awards
- Best Student Paper Award at Web Intelligence 2019,
- National Conference on Artificial Intelligence Excellence Award, National Conference on Artificial Intelligence Student Incentive Award
- MIRU Excellence Award, etc.
Other activities
- Lecturer of “Fundamental of Deep Learning”, “AI Management Endowed Chair”, “World Models and Intelligence”, etc.
- Supervised translation (translation coordination) and shared translation of the book “Deep Learning”