How to Deploy CogVLM to Azure Virtual Machines

Go to your Azure Virtual Machines homepage and create a Virtual Machine in Azure:

How you configure the virtual machine is dependent on how you plan to use the virtual machine so we will not cover specifics in this tutorial.

Roboflow Inference can run on both CPU (x86 and ARM) and NVIDIA GPU devices. But, you will need deploy a system with a GPU to deploy CogVLM. Choose an Azure deep learning operating system when you deploy your system. These operating systems often come with pre-built drivers for use in deep learning tasks like running multimodal models.

When your virtual machine is ready, a pop up will appear. Click “View resource” to view the virtual machine. Or, go back to the Virtual Machines homepage and select your newly-deployed virtual machine.

To sign into your virtual machine, first click “Connect”. Choose the authentication method that you prefer to log into your server. When you have logged in, you are ready to move on to the next step.

‍

Create a new Python file and add the following code:

import base64
import os
from PIL import Image
import requests

PORT = 9001
API_KEY = ""
IMAGE_PATH = "forklift.png"


def encode_base64(image_path):
    with open(image_path, "rb") as image:
        x = image.read()
        image_string = base64.b64encode(x)

    return image_string.decode("ascii")

prompt = "Read the text in this image."

infer_payload = {
    "image": {
        "type": "base64",
        "value": encode_base64(IMAGE_PATH),
    },
    "api_key": API_KEY,
    "prompt": prompt,
}

results = requests.post(
    f"http://localhost:{PORT}/llm/cogvlm",
    json=infer_payload,
)

print(results.json())

This code will make a HTTP request to the /llm/cogvlm route on your Inference installation. This route accepts text and images which will be sent to CogVLM for processing. This route returns a JSON object with the text response from the model.

Above, replace:

1. ROBOFLOW_API_KEY with your Roboflow API key. Learn how to retrieve your Roboflow API key.

3. image.png with the image that you want to use to make a request.

4. prompt with the question you want to ask.

Let’s run the code on the following image of a forklift and ask the question “Is there a forklift close to a conveyor belt?”:

Here is an example output from the script above:

{'response': 'yes, there is a forklift close to a conveyor belt, and it appears to be transporting a stack of items onto it.', 'time': 12.89864671198302}