The main goal of this repository is to challenge different LLMs or, in other words, try to discover the flaws of each of them. In particular we will be testing ChatGPT, Bard, Mistral and LLaMA. In order to do this properly, prompt engineering must be applied correctly to each of them.
LLMs are a subset of machine learning models designed to understand and generate human-like text based on the data they have been trained on. Here are some characteristics:
- Architecture: they use deep neural network architectures. DNNs are a type of artificial neural network with multiple layers between the input and the output layers. These intermediate layers allow DNNs to model and process complex patterns in large datasets.
- Size: it is referred to as "large" because it has billions of parameters. These are the weights and biases in the neural network that are adjusted during training.
- Training: training such models requires vast amounts of text data. This is key, since their outputs will depend on what they have "seen" before.
- Limitations: they do not understand text in the way humans do; they can produce incorrect or nonsensical answers...
In order to test properly how a LLM performs, it is a must to know how to correctly prompt it. Prompting is any form of text, question, information, or coding that communicates to AI what response you are looking for. Each LLM has its own way of communicating, and we, as users, must be able to learn this.
Cisco ExamFolder: This folder contains two Excel files that detail the results and questions related to a set of Cisco certification-style exam problems.ExperimentsFolder: This folder contains the code generated by different LLMs when prompted with a problem related to optical networks.prompt_transformer.py: Basic interface to generate prompts for different types of LLMs.