Use Instructions with Google Gemini

Instroduction

Based on my test, the Instructions to direct ChatGPT's behavior also work with Google Gemini.!

Let's try it out with JavaScript!

Install the Google Cloud SDK

Google Cloud SDK is requried to authenticate the API calls.

Follow the official guide to install it.

brew install --cask google-cloud-sdk

Authenticate the SDK

You'll need to authenticate the SDK with your Google account to use the API. And please make sure you have the Google Cloud project created and billing enabled before you authenticate.

gcloud auth application-default login

Install Dependencies

Install the @google-cloud/aiplatform package by following the official guide:

npm install https://github.com/googleapis/nodejs-vertexai

Write Code

Create a file named index.js and try the instructions like this:

const { VertexAI } = require('@google-cloud/vertexai');
const fs = require('fs');

// Initialize Vertex with your Cloud project and location
const vertex_ai = new VertexAI({ project: 'your-project-id', location: 'us-central1' });
const model = 'gemini-pro-vision';


// Instantiate the models
const generativeModel = vertex_ai.preview.getGenerativeModel({
  model: model,
  generation_config: {
    "max_output_tokens": 2048,
    "temperature": 0.4,
    "top_p": 1,
    "top_k": 32
  },
});

async function generateContent() {
  const instructions = await getInstructions();

  const req = {
    contents: [
      { 
        role: 'user', 
        parts: [
          // Write your instructions before prompts
          { text: instructions },
          // Write your prompts
          { text: 'Cyberpunk 2077' }
        ] 
      }
    ],
  };

  const streamingResp = await generativeModel.generateContentStream(req);

  const resp = await streamingResp.response;
  console.log(JSON.stringify(resp, null, 2));
  console.log(resp.candidates[0].content.parts[0].text);
};

generateContent();

async function getInstructions() {
  // I stored the instructions in a file
  return fs.readFileSync('./instructions.txt', 'utf8'); 
}

Run the Code

I tested the Google Gemini API with my Stable Diffusion Prompts Crafter, and it works!

node index.js

Results

Description

Generate a vibrant and detailed image of a futuristic cityscape from the video game Cyberpunk 2077. The city should be filled with towering neon skyscrapers, flying cars, and bustling crowds. The overall atmosphere should be dark and gritty, with a strong sense of cyberpunk aesthetics. Use a combination of 3D rendering and digital painting techniques to create a highly realistic and immersive image.

Prompts

cyberpunk, futuristic, cityscape, neon, skyscrapers, flying cars, dark, gritty, realistic, immersive, 3D rendering, digital painting

Negative Prompts

blurry, pixelated, low resolution, unrealistic, cartoonish, amateurish, messy, incomplete, inconsistent, nonsensical