Abstract Machine learning is disruptive. At the same time, machine learning can only
succeed by collaboration among many parties in multiple steps naturally as pipelines in an …

There is a rapidly growing number of large language models (LLMs) that users can query for
a fee. We review the cost associated with querying popular LLM APIs, eg GPT-4, ChatGPT …

Machine learning models are often implemented in cohort with humans in the pipeline, with
the model having an option to defer to a domain expert in cases where it has low confidence …

With a growing demand for adopting ML models for a variety of application services, it is vital
that the frameworks serving these models are capable of delivering highly accurate …

There is a rapidly growing number of open-source Large Language Models (LLMs) and
benchmark datasets to compare them. While some models dominate these benchmarks, no …

Training a supervised neural network classifier typically requires many annotated training
samples. Collecting and annotating a large number of data points are costly and sometimes …

Commercial ML APIs offered by providers such as Google, Amazon and Microsoft have
dramatically simplified ML adoptions in many applications. Numerous companies and …

Post-deployment monitoring of ML systems is critical for ensuring reliability, especially as
new user inputs can differ from the training distribution. Here we propose a novel approach …

The Machine Learning as a Service (MLaaS) market is rapidly expanding and becoming
more mature. For example, OpenAI's ChatGPT is an advanced large language model (LLM) …

Machine learning (ML) prediction APIs are increasingly widely used. An ML API can change
over time due to model updates or retraining. This presents a key challenge in the usage of …