GOOGLE GEMINI - MULTIMODAL LIVE CHAT ====================================

GOOGLE GEMINI - MULTIMODAL LIVE CHAT - ELEMENT DESCRIPTION
------------------------------------------------------------------------------
 GOOGLE GEMINI - MULTIMODAL LIVE CHAT provides realtime voice and video conversation capabilities.

STEP-BY-STEP SETUP
--------------------------------
 0) Register on Google Maker Suite to get your API Key https://makersuite.google.com/app/apikey

 1) Try GOOGLE GEMINI LIVE CONVERSATION on https://aistudio.google.com/app/live to confirm your account has access to the required resources.

 2) Implement ERROR workflow as per the demo so any errors are caught to your app.

 3) Register on plugins.wiseable.io. Create a new Credential which associates your BUBBLE APP URL,  GOOGLE GEMINI API KEY.
  The registration service will generate your PUBLIC ACCESS KEY. This key serves as a secure proxy for your real API key. It allows your application to communicate with the service without exposing your real API key. Since this PUBLIC ACCESS KEY is explicitly tied to your registered BUBBLE APP URL, it can only be used from that domain, ensuring that even if the key is publicly visible, it remains safe and cannot be misused by unauthorized sources.

 4) In the Plugin Settings, enter your PUBLIC ACCESS KEY generated at the previous step.

 5) Add the GOOGLE GEMINI - MULTIMODAL LIVE CHAT ELEMENT to the page on which the chat must be integrated. Select the RESULT DATA TYPE as CONVERSATION (GOOGLE GEMINI - MULTIMODAL LIVE CHAT).

 6) Integrate the logic into your application using the following GOOGLE GEMINI - MULTIMODAL LIVE CHAT element's states and actions:

 FIELDS:
   - RESULT DATA TYPE: Must always be selected as CONVERSATION (GOOGLE GEMINI - MULTIMODAL LIVE CHAT).      
   - CAMERA NAME: Name of the Camera to use as a source. It must be one of the item of INPUT CAMERAS state.
   - MICROPHONE NAME: Name of the Microphone to use as a source. It must be one of the item of INPUT MICROPHONES state.
   - MODEL NAME: Name of the model. See https://ai.google.dev/gemini-api/docs/models#gemini-2.0-pro.
   - ROLE INFORMATION: Define the role of the model.
   - ECHO CANCELLATION : Echo cancellation is a feature which attempts to prevent echo effects on a two-way audio connection by attempting to reduce or eliminate crosstalk between the user's output device and their input device.
   - NOISE SUPPRESSION : Noise suppression automatically filters the audio to remove background noise, hum caused by equipment, and the like from the sound before delivering it to your code.
   - AUTO GAIN CONTROL : Automatic gain control is a feature in which a sound source automatically manages changes in the volume of its source media to maintain a steady overall volume level.
   DATA RETRIEVAL SETTINGS
   - GOOGLE SEARCH : Set to yes to activate Google Search when needed to answer to the user prompt.    

 EVENTS:
 - ERROR: Event triggered when an error occurs.
 - CALL TOOL: Event triggered when a function is triggered, as defined in TOOLS of START CONVERSATION action.
 - CONVERSATION TO SAVE: Event triggered when any of the conversation has changed.
 - CONVERSATION READY TO START: Event triggered when a conversation is ready to start.

 EXPOSED STATES:
 Use any element able to show/process the data of interest (such as a Group with a Text field) stored within the result of the following states of the GOOGLE GEMINI - MULTIMODAL LIVE CHAT ELEMENT:
 - ERROR: Error message upon Error event trigger.
 - INPUT CAMERAS : List of detected cameras, populated after DETECT DEVICES action.
 - INPUT MICROPHONES : List of detected microphones, populated after DETECT DEVICES action.
 - IS LISTENING: Returns true when listening is in progress.
 - IS AI WATCHING CAMERA: Returns true when camera watching is in progress.
 - IS AI WATCHING SCREEN: Returns true when screen watching is in progress.
 - IS AI SPEAKING: Returns true when AI speaking is in progress.
 - TOOL NAME: Name of the tool, set upon CALL TOOL event.
 - IS CAMERA ON : Returns true when camera is on.
 - IS SCREEN SHARING ON : Returns true when screen sharing is on.

 - TOOL ARGUMENTS: Arguments of the tool, set upon CALL TOOL event.
 - TOOL CALL ID: Unique identifier of the tool call, set upon CALL TOOL event.
 - ALL CONVERSATIONS: List of conversations containing a list of role and message content.
 - ALL CONVERSATIONs (RAW DATA): String containing All Conversations in JSON format. You may use this string to load threads in "LOAD CONVERSATIONS" action.


 ELEMENT ACTIONS - TRIGGERED IN WORKFLOW:
   - DETECT DEVICES: Detect both audio and video input devices. Populate INPUT CAMERAS and INPUT MICROPHONES states.
   - START CONVERSATION: Start Voice Conversation.
     Inputs Fields:
       - VOICE: The voice to use when generating the audio. Valid values: Puck | Charon | Kore | Fenrir | Aoede.
       - RESUMED CONVERSATION ID: Must be one of the CONVERSATION ID of ALL CONVERSATIONS state.
       - TEMPERATURE: Sampling temperature to use, between 0 and 2. Higher values mean the model will take more risks. Try 0.9 for more creative applications, and 0 for ones with a well-defined answer.
       - TOOLS: Array containing function definitions. See https://ai.google.dev/gemini-api/docs/function-calling/tutorial?lang=node#create-function-declarations.
   - SEND TEXT MESSAGE : Send a message in text format.
     Inputs Fields:
       - MESSAGE : The message in text format.        
  - STOP CONVERSATION: Stop Voice Conversation.
     Inputs Fields:
       - INTERRUPT PLAYBACK : If true, immediately interrupt playback. If false, buffered audio will continue playing until completed..
  - START SCREEN SHARING: Start screen sharing.
  - STOP SCREEN SHARING: Stop screen sharing.
  - START CAMERA: Start camera.
  - STOP CAMERA: Stop camera.
  - SEND SCREENSHOT OF ELEMENT ID: Send the screenshot of an element on the same page with the specified Element ID.
      - ELEMENT ID : Send the screenshot of an element on the same page with the specified Element ID
  - SET CONVERSATION TITLE : Set a custom Conversation Title of a given Conversation ID.
     Inputs Fields :
       - CONVERSATION ID : Set the title to this Conversation ID. A valid Conversation ID is one of the exposed state ALL CONVERSATIONS.    
  - DELETE CONVERSATION ID : Delete the specified CONVERSATION ID.
     Inputs Fields :
       - CONVERSATION ID : Delete the specified CONVERSATION ID.        
  - LOAD CONVERSATIONS : Load the conversations.
     Inputs Fields :
       - CONVERSATIONS (RAW DATA) : String formatted as JSON-safe, containing the conversations in JSON format.
  - DELETE CONVERSATION ID : Delete the specified CONVERSATION ID.
     Inputs Fields :
       - CONVERSATION ID : Delete the specified CONVERSATION ID.      
     Inputs Fields:
  - SEND TOOL RESULT: Send the tool results. See https://ai.google.dev/gemini-api/docs/function-calling/tutorial?lang=node#create-function-declarations.
     Inputs Fields:
       - TOOL CALL ID: Unique identifier of the tool call from the TOOL CALL ID state, which is set upon CALL TOOL event.
       - TOOL RESULT: Result of the tool to pass.

IMPLEMENTATION EXAMPLE
======================
Feel free to browse the app editor in the Service URL for an implementation example.

TROUBLESHOOTING
================
Any plugin-related error will be posted to the Logs tab, "Server logs" section of your App Editor.
 Make sure that "Plugin server side output" and "Plugin client side output" are selected in "Show Advanced".

> Server Logs Details: https://manual.bubble.io/core-resources/bubbles-interface/logs-tab#server-logs

PERFORMANCE CONSIDERATIONS
===========================
 N/A

QUESTIONS?
===========
Contact us at [email protected] for any additional features you would require or support questions.