Ameer Haj Ali

Ameer is the Head of Platform and Infrastructure Engineering organizations at Anyscale, Inc (A $1.1B startup that provides a unified compute platform for running any AI applications. Here is me demoing our product) where he led some of the most critical projects, such as Anyscale General Availability, Anyscale's new AI infrastructure, LLM Serving & Endpoints, and Faster, Cheaper & Highly Available LLM inference. The Platform team is responsible for Ray Serve: the fastest growing open-source ML production serving library and builds everything related to the lifecycle of Ray workloads: Ray Client, Jobs, highly-available Services, observability, running ray locally, and runtime environments.
The infrastructure team builds the infrastructure for cluster orchestration, billing, monitoring, autoscaling, which Anyscale and its customers run on. The team also maintains the Ray Cluster and Autoscaler (video), Ray Client, Cloud providers, and Ray on Kubernetes in open-source Ray. The team also develops the core engine of Anyscale’s product for providing the infinite laptop serverless experience, used by every user of Ray and the product. 

Ameer completed his Ph.D. in two years (summa cum laude, the fastest in the department) Electrical Engineering and Computer Science at UC Berkeley in the ADEPT Lab and RISE Lab, where he was advised by Professors Ion Stoica and Krste Asanovic. His research focused on Compiling, Auto-Tuning, Code Optimization, Machine Learning, Reinforcement Learning, and hardware for machine learning. At UC Berkeley Ameer helped bring up/led many projects spanning machine learning in compiler optimization and hardware software codesign. This includes Gemmini, AutoPhase, NeuroVectorizer, ProTuner, Ansor, AutoCkt and more. 

Ameer finished his M.Sc. studies (summa cum laude, the valedictorian) at the Technion in 2018, where he worked on using emerging memory technologies to enhance the performance of modern computer systems with Professor Shahar Kvatinsky and made multiple journal and conference publications. He also finished four years of undergraduate studies in computer engineering at the Technion in only three years, graduating summa cum laude and receiving the valedictorian honor. During Summer 2019, Ameer worked as a research scientist at Intel Labs in the Brain Inspired Computing Lab where he explored deep reinforcement learning in system optimization, and built NeuroVectorizer and RLDRM (awarded best paper award). During his undergraduate studies, Ameer worked at Mellanox Technologies as a chip designer, focusing on creating design and automation tools that facilitated the formal and dynamic verification process.

Ameer was granted O1 and EB1 (Einstein) US Visas, which are granted for an individual who possesses extraordinary ability.

In his free time, Ameer volunteers as a board member on the board of directors of American Technion Society (ATS), and promotes the underprivileged Arab minority in Israel.

Contact

Education

Ph.D. student, working with Prof. Krste Asanovic and Prof. Ion Stoica.

Finished 5-year track summa cum laude in two years.

Thesis: Machine Learning in Compiler Optimization

Research Interests: Compiling, Auto-Tuning, Code Optimization, Machine Learning, Reinforcement Learning, and hardware for machine learning.

​​

M.Sc., Electrical Engineering.

Finished 4-year track summa cum laude (top 3%) in three years.

Member of the President’s List of highest honors for excellent scholastic achievements every semester.


B.Sc., Computer Engineering.

Finished 4-year track summa cum laude (top 3%) in three years.

Member of the President’s List of highest honors for excellent scholastic achievements every semester.

Professional Experience

    Industry

- Leading the cloud platform engineering (1 Tech Lead/Architect, 2 PMs, 2 Managers, 11 L3-L6 SWEs), which builds the foundational blocks of Anyscale's serverless infrastructure end-to-end. This includes cluster orchestration, autoscaling, logging, metrics, billing, and a multi-cloud, multi-region architecture that provides a reliable and scalable managed Ray experience for Anyscale customers.

- Project manager of Anyscale's All-In-Our-Account (AIOA) and All-In-Customer-Account (AICA) projects. AIOA is a new K8s-based infrastructure that is completely managed by Anyscale (Anyscale Cloud) that customers can run on. AICA is a new K8s-based infrastructure that is managed by Anyscale but runs in the customer's account.

A demo video of the final product was broadcasted in the Ray Summit 2021.

- led the Serverless team(7 Eng, 1 PM, ex. Google/Uber/MS/FB/Amazon/etc), which develops and maintains the Ray Cluster and Autoscaler (video), Ray Client, Cloud providers, and Ray on Kubernetes. The team develops the core engine of Anyscale's product for providing the infinite laptop serverless experience, used by every user of Ray and the proprietary product.

- led the development of C++ API for Ray to allow Anyscale’s users to run distributed C++ applications.  This is being used by multiple companies including Intel, Ant Financial, and ByteDance.

- led multiple projects requested by Anyscale’s early customers to get them on board. For example, I built a cluster management system to allow them to run Ray on their on-premise clusters

- My team works closely with customers to address all their needs and concerns as fast as possible.

- led the development of features that reduced the company’s annual cloud providers’ bill by more than $0.5M (growing linearly with the number of employees).


- Led the development of the open source Ray K8s operator.

- Implemented scalable Scikit-learn on top of Ray that runs on large clusters.

- Lead the implementation of C++ client API for Ray.

- Built a cloud gateway that enables plugging any type of remote clusters including on-prem.

- Contributed to RLlib: scalable reinforcement learning library.

- Directed the first Ray meetup 2019 in Israel.


-  A View on Deep Reinforcement Learning in System Optimization.

- RLDRM: Closed Loop Dynamic Intel RDT Resource Allocation with Deep Reinforcement Learning.


    Graduate Teaching Assistance

University of California, Berkeley, ​2019 - present

Technion, 2015 - 2018

Awards and Fellowships

Advised Students

University of California, Berkeley

Israel Institute of Technology, Technion

Publications

Ameer Haj-Ali.

[thesis]

58th ACM/ESDA/IEEE Design Automation Conference (DAC 2021), December 2021. [paper]


Ameer Haj-Ali, Hasan Genc, Qijing Huang, William Moses, John Wawrzynek, Krste Asanović, Ion Stoica.

arXiv preprint, 2020. [paper]

Lianmin Zheng, Chengfan Jia, Minmin Sun, Zhao Wu, Cody Hao Yu, Ameer Haj-Ali, Yida Wang, Jun Yang, Danyang Zhuo, Koushik Sen, Joseph Gonzalez, Ion Stoica.

The 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2020. [paper]


Ameer Haj-Ali, Nesreen Ahmed, Ted Willke, Sophia Shao, Krste Asanovic, Ion Stoica.

International Symposium on Code Generation and Optimization (CGO), 2020. [paper][code][video]


Ameer Haj-Ali*, Qijing Huang*, William Moses, John Xiang, Krste Asanovic , John Wawrzynek, Ion Stoica.

Proceedings of Machine Learning and Systems (MLSys), 2020. [paper][code][video]


Keertana Settaluri, Ameer Haj-Ali, Qijing Huang, Suhong Moon, Kourosh Hakhamaneshi, Ion Stoica, Krste Asanovic, Borivoje Nikolic.

Design Automation and Test in Europe (DATE), 2020. [paper][code]


Bin Li, Yipeng Wang, Ren Wang, Charlie Tai, Ravi Iyer, Zhu Zhou, Andrew Herdrich, Tong Zhang, Ameer Haj-Ali, Ion Stoica, Krste Asanovic.

IEEE Conference on Network Softwarization (NetSoft), 2020. [paper][code]


Ameer Haj-Ali, Nesreen Ahmed, Ted Willke, Joseph Gonzalez, Krste Asanovic, Ion Stoica.

arXiv preprint, 2019. [paper]


Ameer Haj-Ali, Nesreen Ahmed, Ted Willke, Sophia Shao, Krste Asanovic, Ion Stoica.

Workshop on ML for Systems at NeurIPS, 2019. [paper][code]


Ameer Haj-Ali*, Hasan Genc*, Vighnesh Iyer*, Alon Amid*, Howard Mao, John Wright, Colin Schmidt, Jerry Zhao, Albert Ou, Max Banister, Yakun Sophia Shao, Borivoje Nikolic, Ion Stoica, Krste Asanovic.

arXiv preprint, 2019. [paper][code]


Ameer Haj-Ali*, Qijing Huang*, William Moses, John Xiang, Ion Stoica, Krste Asanovic , John Wawrzynek.

FCCM, 2019. [paper][code][video]


Ameer Haj-Ali. 

M.Sc. Thesis. [thesis


Ameer Haj-Ali, Ronny Ronen, Rotem Ben-Hur, Nimrod Wald, and Shahar Kvatinsky.

Elsevier, 2020. [chapter]


Nishil Talati, Rotem Ben-Hur, Nimrod Wald, Ameer Haj-Ali, John Reuben, and Shahar Kvatinsky. 

Springer, 2020. [chapter]


Rotem Ben-hur, Ronny Ronen, Ameer Haj-Ali, Debjyoti Bhattacharjee, Adi Eliahu, Natan Peled, and Shahar Kvatinsky.

TCAD, 2019. [paper]


Tzofnat Greenberg-Toledo, Roee Mazor, Ameer Haj-Ali, and Shahar Kvatinsky.

TCAS-I, 2019. [paper]


Ameer Haj-Ali, Rotem Ben-Hur, Nimrod Wald, Ronny Ronen, and Shahar Kvatinsky.

IEEE Micro, 2018. [paper]


Ameer Haj-Ali, Rotem Ben-Hur, Nimrod Wald, Ronny Ronen, and Shahar Kvatinsky.

TCAS-I, 2018. [paper]


Ameer Haj-Ali, Rotem Ben-Hur, Nimrod Wald, and Shahar Kvatinsky.

ISCAS, 2018. [paper]


Nishil Talati, Ameer Haj-Ali, Rotem Ben-Hur, Nimrod Wald, Ronny Ronen, Pierre-Emmanuel Gaillardon, and Shahar Kvatinsky.

DATE, 2018. [paper]


John Reuben, Rotem Ben-Hur, Nimrod Wald, Nishil Talati, Ameer Haj-Ali, Pierre-mmanuel Gaillardon, and Shahar Kvatinsky.

PATMOS, 2017. [paper]


John Reuben, Rotem Ben-Hur, Nimrod Wald, Nishil Talati, Ameer Haj-Ali, Pierre-Emmanuel Gaillardon, and Shahar Kvatinsky.

Springer, 2017. [chapter]