New AI Research Presents AttrPrompt: An LLM as a Training Data Generator for a New Paradigm in Zero-Shot Learning

The performance of large language models (LLM) has been impressive in many different natural language processing (NLP) applications. In recent studies, LLMs have been proposed as generators of task-specific training data to reduce the need for task-specific data and annotations, particularly for text classification. While these efforts have demonstrated the utility of LLMs as data producers, they have largely focused on improving the training phase, when the data generated is used to train task-specific models, while leaving the data creation process intact. Mountain. To query LLMs, the prevailing method uses a single-class conditional prompt, which can reduce the variety of data provided and perpetuate the inherent systematic biases of LLMs.

A new study by Georgia Tech, the University of Washington, UIUC and Google Research analyzes four difficult tasks of classifying subjects with large cardinality from different domains. Link LLM to ChatGPT for its ability to write high quality human language. The team primarily uses data attributes to assess the level of bias and diversity within the created training set. Specifically, data attributes consist of several attribute dimensions and various attribute values, each of which represents one possible realization of the attributes themselves.

The researchers used a trained attribute classifier to analyze attribute bias in the data set generated by SimPrompt. They investigate how different attributes can influence the final results of a model. To generate attributed data, they use ChatGPT and add constraints to questions with certain values ​​for the required characteristics. The researchers find that models trained on datasets generated with random characteristics perform significantly better than those trained on datasets with fixed attributes, highlighting the importance of attribute variation in the generated dataset.

The team suggests generating data using differently attributed prompts to reduce attribute bias and increase the attribute diversity of the generated data. Using the LLM, an interactive and semi-automated process is first initiated to determine the appropriate attribute dimensions and values ​​for a given classification task. The standard class conditional prompt for LLM data queries is then replaced by more complex prompts generated from randomly combined properties. They coined the term AttrPrompt to describe these various attributable triggers.

The researchers empirically evaluate the datasets created on the four classification tasks by comparing the results of models trained in two scenarios: 1) on the generated dataset only and 2) on a merged dataset, including the original training set and the generated set. The dataset created using AttrPrompt works much better than the dataset created with SimPrompt in both cases. Their results also show that AttrPrompt is superior to SimPrompt in data/budget efficiency and flexibility across a broad range of model dimensions and LLM-as-training-data-generator strategies.

AttrPrompt is notable because it provides the same performance as SimPrompt while requiring only 5% of the query cost of ChatGPT required by SimPrompt. Finally, they show for the first time that AttrPrompt beats SimPrompt in all evaluation criteria by extending the LLM-as-training-data-generator paradigm to the most difficult multi-label classification problems.

Check out thePaper ANDGithub link.Don’t forget to subscribeour 25k+ ML SubReddit,Discord channel,ANDEmail newsletter, where we share the latest news on AI research, cool AI projects, and more. If you have any questions regarding the above article or if you have missed anything, please do not hesitate to email us atAsif@marktechpost.com

Featured tools:

Check out 100s AI Tools in the AI ​​Tools Club