Summary of recent tweets

Yves-Alexandre de Montjoye, an AI researcher, has been tweeting about the transparency and auditing of Language Models (LLMs) recently. In one tweet, he shared a great piece in @lemondefr discussing their method to detect whether an LLM has seen a book during training. The paper associated with this work is also available on arXiv. Yves-Alexandre collaborated with researchers from Imperial College for this project.

In another tweet, Yves-Alexandre highlighted his recent research that helps detect whether a literary or scientific text was read by an LLM. He spoke to @lecho about recent trends in the AI space and emphasized the need for tools to audit LLMs and understand what they learn and where they learn from.

Yves-Alexandre expressed concern over relying solely on companies or their boards to self-regulate AI in a tweet related to OpenAI events. He mentioned that separate teams have developed data provenance tools to check the datasets used for training models publicly. Additionally, other researchers have obtained similar results on Wikipedia data, showing that LLMs do memorize information from documents they see during training.

The transparency of LLM pre-training data is gaining traction as multiple new preprints have been posted on this topic since Yves-Alexandre released his work. These findings raise questions about biases, misinformation propagation, ownership rights, copyright implications, and fair use exemptions related to these models.

Overall, Yves-Alexandre's tweets indicate a positive sentiment towards increasing transparency and auditing mechanisms for Language Models in order to better understand their learning processes.

Trends identified: Transparency and auditing of Language Models (LLMs).