BERT works by pre-training a deep neural network using large amounts of unlabeled text data in an unsupervised manner. During the pre-training process, the model learns to generate contextualized word embeddings, which are representations of words that take into account the context in which they appear.

The pre-training process involves two tasks :

* Masked language modeling
* Sentence prediction.

In masked language modeling, a random subset of the input tokens are masked and the model is trained to predict the original token based on the surrounding context. This forces the model to learn bidirectional representations of language, as it must take into account the context both before and after the masked token in order to predict it.

In next sentence prediction, the model is trained to predict whether two sentences are contiguous in the original text or not. This encourages the model to learn relationships between sentences and better understand the structure of language.

Once the model has been pre-trained, it can be fine-tuned for specific natural language processing tasks by adding a task-specific output layer and training the model on labeled data for that task. The fine-tuning process is typically fast and requires relatively few labeled examples, as the model has already learned general language representations during pre-training.