We already knew that Apple was largely working quietly on AI. However, the new information that emerges reveals Ferret, the company's first artificial intelligence tilt model.

In October, researchers working for Apple and Cornell University quietly released an open-source multimodal large language model (LLM) called "Ferret" that can use regions of images for queries. While Ferret was released as a research release, no announcement was made for its launch. Ferret's code was released with Ferret-Bench on October 30, and checkpoints were introduced on December 14.

Important step for artificial intelligence from Apple

Apple's release of the Ferret model as open source is done under a non-commercial license, so it cannot be commercialized in its current form. However, there is a possibility that the model will be used in some way in a future Apple product or service.

An October tweet from Apple AI/ML research scientist Zhe Gan explains that Ferret is being used as a system that can "reference anywhere and be based on anything in any detail" in an image. It can also do this by using the shape of any region within an image.

In simpler terms, the model can examine a region drawn on an image, identify and define elements that can be used by the user in a query within this region, and draw a bounding box around the detected element.

For example, by highlighting the image of an animal in the image and asking Ferret what the animal is, the model can determine the type of creature and whether the user is referring to a single animal from a group. Ferret can also use the context of other elements detected in the image to offer further answers.

This statement is important because it is a known fact that Apple is a very secretive company about its own technologies. On the artificial intelligence side, Apple seems willing to be more open . On the other hand, Apple also has an infrastructure problem; While it is working to increase the number of AI servers it has, it may not currently have the scale to go toe-to-toe with ChatGPT, for example.

It is stated that Ferret is trained on 8 A100 GPUs with 80GB of memory. This number is very low compared to Apple's competitors. However, it was announced by its CEO, Tim Cook, that Apple spent millions of dollars on artificial intelligence. Therefore, undisclosed projects may also be running in the background. It was also recently reported that Apple started negotiations with news publishers for artificial intelligence training. It looks like Apple plans to use major news publishers to train its AI models. The company is said to have made deals worth at least $50 million for each news organization for several years