Arslan Chaudhry is a Rhodes Scholar studying Artificial Intelligence and Machine Learning towards his Ph.D. dissertation at Oxford. Originally from Lahore, Arslan completed his undergraduate education from UET Lahore in Electrical Engineering and worked with Mentor Graphics, a popular company specializing in embedded electronics, for two years before arriving at Oxford. As part of our series ‘Artificial Intelligence: Truths and Myths’, Spectra spoke to Arslan about his journey, Artificial Intelligence (AI) and his work in this field.
The Interview consists of two parts; the first part being published here, explores Arslan’s personal experience as a student at UET and now at Oxford. In the second part, due to be published next week, we have an in-depth conversation with Arslan on multiple themes of Artificial Intelligence, including, but not limited to, limitations of current AI systems, his work on continual learning, academic lethargy in Pakistan and the need for a national AI policy.
The interview has been condensed and edited for the purposes of clarity and brevity.
Spectra: How did you come into electrical engineering? Was it a personal inspiration or were there any other factors?
Arslan Chaudhry: No [there was no personal inspiration]. I am a product of a system in which creativity or exploration is not encouraged at the primary education level, and if you happen to be a good student then the path is already set for you. If you do well in high school, you have to do FSc. pre-engineering or FSc. pre-medical. And if you do well there too, then you have to go to the university [and choose a major] with the highest merit in the country. Throughout my life, I’ve been following this crowd [and landed in electrical engineering]. So, as such, there was no eureka moment at which I realized, “aha, engineering is something that I really want to do.” But I think ever since I came to UET and started exploring engineering and mathematics, I gradually started becoming interested in these subjects. And then over four years, I tried to read as much as I could and developed an interest in math, engineering, and computer science. But I kept my other interests as well, which are poetry, literature, and politics.
Who were your heroes in high school and how have they evolved over time?
I will stick to academic heroes, as this is slightly easier to answer. Growing up, I used to love mathematicians. I was always inspired by all these romantic stories of [cognition of] really simple yet powerful ideas. For instance, you take Gauss and his story about how he came up with the formula of summing up integers from one to 100; John Nash and how he came up with Nash equilibrium and so on. I think mathematicians were my first love, particularly [Carl Freidrich] Gauss and [Kurt] Godel. I was absolutely mesmerized by [Godel’s] incompleteness theorem. Then I remember watching Andrew Wiles documentary on proving Fermat’s last theorem and how inspired I felt after that. There’s also a mathematician who spent eight years on a problem; first five or six years on the problem and which was proven wrong. Then he spent three more years and so on. So yeah Gauss, Godel, [John] Nash, Andrew Wiles, these were my heroes growing up. And then these days, since I’m working on machine learning and AI and computer vision, I’ll be a bit conservative and stick with the usual suspects. So it will be Geoff Hinton, Yann LeCun, and Yoshua Benjio, and in computer vision, Alyosha Efros, Jitendra Malik and Andrew Zeissman; the kind of work that Andrew is producing at this age and the frequency with which he is publishing is just remarkable. Then Geoff, Yann, and Yoshua. These are my heroes these days. And one other person who has constantly inspired me throughout my life is Faiz Ahmed Faiz, a revolutionary Urdu poet. Faiz has always been my inspiration. I try to read Faiz myself and I also force my friends as well.
Do you have any favorite Faiz couplets?
Interesting question. Here’s one:
دربار میں اب سطوت شاہی کی علامت
درباں کا عصا ہے کہ مصنف کا قلم ہے
So, in this couplet, Faiz is saying it is not clear what the symbol of majesty is these days. Is it ‘Darban ka asa’[salute of guardsmen] or is it ‘musanif ka qalam’[pen of a writer/thinker]? So, essentially Faiz is criticizing the journalists here and I think that this is very pertinent to what is happening in Pakistan these days where media and press which can speak truth to power is conspicuously missing.
How would you describe your experience as a student at UET?
I think it’s been a fun experience in lots of ways. Unfortunately, the educational experience was not good for me, and this was not something I was expecting of UET, to be very honest. The only reason I went to UET was that [because it is the top engineering institutions of the country] I [expected that I] would be surrounded by top students from all over Punjab. But then I always knew that I will have to study myself. And that is what I did in the four years there. I used to go to the classes just for the sake of attendance and study on my own.
There was also no research being done at UET. Even at the department, I was part of, Electrical Engineering [which was the top department at UET], I can hardly think of any professor there who was active in research. The sports facilities were just fine. So, in conclusion, it was a subpar experience overall; academic quality was not the hallmark of UET, and it was the people I surrounded myself with that made it a pleasant experience for me.
In comparison to Oxford, where do you think UET stands in terms of teaching quality and research?
I am slightly handicapped in answering this question because even though I have audited a lot of courses at Oxford, I have not been part of regular classes. Whereas in UET, I was a part of regular classes but I was not part of any research group if there were any. There were KICS and CLE, but the professors working there were either not from our department or the kind of problems they were working on were not really tied to things that we were studying in classes. But still, I can say that research at Oxford is a lot better than research at UET. You can find all kinds of people here [at Oxford] who are actually working on cutting edge areas in their field. And then there are visitors from Stanford, Berkeley, and other top universities and from big companies like DeepMind, FAIR who come and comment on your work and give you suggestions. But being in computer science and especially in the machine learning field, I have realized in the last three years that the kind of research I am doing here, I could easily have done that in UET provided enough computational resources and other supplementary things. But the advantage here is the network that you get to build and the kind of people that you can interact with face-to-face on a daily basis. You can replicate Oxford’s infrastructure in UET as well, but the key component which is lacking in UET is basically the right people at the moment.
What is the key determinant due to which Oxford has these kinds of people, and UET does not? Is it funding or is it a prestige factor?
I think it’s a mixture of all these things. The main thing is that at UET, students do not have any incentives to do research, especially in their undergrad. However, in Oxford, all the students in their final year project, which they call FYP, normally come and work with one of the labs here in Oxford. So, for example, in our lab, Torr Vision Group, we take one or two masters or undergrad students every year and then they work with one of the postdocs or one of the senior Ph.D. students and then again next year. So, it’s a cycle basically. So, once you see that there are people in your cohort, who are doing research, it creates an incentive for other people as well. But if you see that none of the people in your cohort are working on any kind of research problem or research projects [like in UET], there is little incentive for you to get involved in the research. And this has to do with faculty as well. So here, faculty actively encourages students to come and work on these research projects in their lab, because practically every faculty member has his or her own lab. And believe it or not, people in the universities in US/UK, they prefer to do research than to teach but in our part of the world, once you get the tenure you are happy teaching because that’s the easiest thing that you can do. You prepare notes only once and then you keep on teaching that course, again and again. Whereas if you have to perform research then you have to write grant proposals, you have to come up with new projects, you have to engage your students on these projects, which is time-consuming and takes a lot of energy. And of course, finally, it has to do with funding as well. Computer Science funding is generally an issue for the developing world, but I’m quite sure that the kind of projects that we need to be working on, in our parts of the world, we can easily get funding for, provided we are willing to put in some effort and time, which we are not always into.
Do you think that at the university level in Pakistan, there should be more focus on machine learning?
Yes, definitely. So in the IITs [Indian Institute Of Technology], they already have the basic courses, like probability, optimization and so on. But they also have courses like Introduction to Computer Vision and Introduction to Natural Language Understanding or Introduction to Natural Language Processing using PyTorch or TensorFlow. They are already offering these courses which means that they are already writing the code and implementing projects in machine learning and so on. In contrast, when I was in UET, there was no formal coursework for machine learning or there was no stream if you wanted to specialize in machine learning. Introduction to Machine Learning was not offered at our time. And more than that, I think we should take a step beyond Intro to Machine Learning. Right now, we have three streams at UET in Electrical Engineering: power, telecommunication/electronics, and computers. I think it is high time that we have another stream for Applied Mathematics/Machine Learning/Scientific Computing within EE because there are so many students there who are mathematically inclined but are not given an opportunity to pursue that area and machine learning can then be taught in that stream.
What is your advice for someone entering the ML/AI field right now, in particular, a Pakistani engineering undergraduate looking to specialize in machine learning?
Yes, I think it is absolutely necessary to do the basic coursework. So, anyone who is genuinely interested in machine learning ought to do courses like linear algebra, probability, optimization, stochastic processes because these courses are so fundamental and a must for excelling in machine learning. And if you are not satisfied with the quality of these courses at your university, you can always go online and find great resources. Gilbert Strang lectures on Linear Algebra are like a goldmine, then there is a course on convex optimization by Stephen Boyd and in machine learning, there is Andrew Ng’s course, and then there are CS231 and CS224 by Stanford. Also, there is a Nandos De Freitas course on deep learning that he taught at Oxford in collaboration with DeepMind, that is also good. There is another Nandos De Freitas course that he taught at UBC, the University of British Columbia, that is also good. So, you can always go online and find enough resources to start learning.
And nowadays, it is so easy to access all these cool tool chains. So for example, you have PyTorch and you have TensorFlow and then you have GPUs on Google Collab and so on. You can easily go and implement whatever you want. And there is enough code available out there that you can just download and try. So, do the coursework, try to work on as many problems and projects as you can and keep on reading these new papers because machine learning is such a high paced field. In our undergrad studies, we are normally not encouraged to read research papers, at least I don’t remember reading them very often. Whereas, I must remind you that most of the machine learning, or at least this new form of machine learning which is deep learning, is basically based on research papers. There is no formal textbook, there is actually one by Goodfellow, Bengio and Aoron Courville but that again just gives you a very high-level overview of the field. You need to find your niche, like whichever field you’re interested in: language processing or vision or robotics or reinforcement learning or generative models and so on. And then start reading the research papers which are coming out in that area and try and implement as many papers as you can. Because the problem is that once you graduate, and you start applying to these Ph.D. positions or these graduate-level positions, you have professors who are invariably going to ask you about the recent trends in deep learning and machine learning. Like, have you read this paper? What do you think about this paper? What do you think about the lottery ticket hypothesis? These are all things that any professor is going to ask you because they are not expecting someone who’s just an undergrad. They’re expecting someone who has done a good enough amount of work in machine learning. So nowadays, the threshold to enter into a top-notch Ph.D. program in machine learning is very high and especially if you’re competing with the IITs and with students from East Asia who are already studying all these courses. So you cannot expect to come into your Ph.D. program and spend one or two years just researching the problem and just learning how to code, how to build projects, how to use TensorFlow, how to use Pytorch or what is a deep network or how does back-propagation work. These are things you ought to know before starting a Ph.D. program.
And what would be your advice to a high schooler interested in Artificial Intelligence and computer science in general?
I think as a high schooler I would probably start with programming because that is something which we don’t do in high schools. So I think one should learn to program instead of starting on AI. I mean, you can always start on AI, I’m not discouraging that but if I were to go back to high school or junior high again, then I would probably start programming a bit earlier. And then inculcating this notion of how to program and what does programming imply? For this again, there are a bunch of courses online for beginners like Python for kids, or C for kids that you can start doing. You can start with very simple programs that give you a notion of how an algorithm works and how it crunches numbers and gives you an output. So programming is absolutely necessary. Once you know to program, then you can implement simple math problems. And that will allow you to appreciate the beauty of machines to crunch numbers and do something beautiful. So, start programming early.