Setup Guide for Scispot Agent
Introduction
This document serves as a guide for setting up the Scispot Agent on your system. You can create and configure your Scispot Agent within Scispot Glue. Follow the instructions relevant to your system.
Prerequisites
Before proceeding with the installation, ensure that you have a shared local folder that the Scispot Agent can access. This folder will be used by the agent to listen for changes. Additionally, the machine on which you are setting up the agent requires internet access to communicate with the Scispot cloud.
Java Runtime Environment Requirement
To run the Scispot Agent, your machine must have Java Runtime Environment (JRE) version 11 installed and properly configured. If you do not have JRE 11, you will need to install it.
Installing Java Runtime Environment 11
Download JRE 11: Visit the official Oracle website or your preferred vendor’s site to download the Java Runtime Environment 11 installation package suitable for your operating system.
Install JRE 11: Run the downloaded installation package and follow the on-screen instructions to install JRE 11 on your machine.
Configuring Java Runtime Environment 11
After installation, you may need to configure your system to use the installed JRE 11 as the default Java platform.
For Linux and macOS:
Set JAVA_HOME: Set the JAVA_HOME environment variable to point to the installation directory of JRE 11.
Update PATH: Add $JAVA_HOME/bin to your PATH environment variable to access Java commands from any terminal session.
For Windows:
Set JAVA_HOME: Set the JAVA_HOME environment variable through the System Properties -> Environment Variables -> System Variables panel, pointing to the JRE 11 installation directory.
Update PATH: Add %JAVA_HOME%\bin to your system’s PATH variable so that Java commands are available from any command prompt window.
Installation Steps
To use Scispot Agent, first you need to create an agent inside Scispot Glue and install it. Follow these steps to set up the Scispot Agent:
Go to Scispot Glue.
Click on Agents at the top.
Click on Add Agent on the left side.
Add a name for the agent and select your operating system to download it.
Internet Access Requirement
The machine where the Scispot Agent is installed must have internet access. The agent communicates with the Scispot cloud and will send data to it. Ensure that your firewall or network settings do not block this communication.
Understanding the Monitoring Mechanism
The Scispot Agent uses a Java service that employs the WatchService API to monitor for changes in the file system within the specified directory path. It listens for ENTRY_CREATE and ENTRY_MODIFY events to detect new or updated files and implements a debounce logic to handle these events appropriately.
Running the Agent
Once the script file has been correctly edited and saved, execute the script to start the agent. It will begin monitoring the specified folder for any changes and perform actions as necessary.
Optional for FTP connections only: Enabling Local FTP Connection for Instrument Integration
In addition to monitoring file system changes, the Scispot Agent can be configured to facilitate FTP connections for integrating various instruments that require such connectivity. This feature allows instruments to directly communicate with the Scispot Agent via FTP, enabling seamless data transfer and synchronization.
Configuring Your FTP Client
To establish an FTP connection with the Scispot Agent, follow these steps:
Set the Connection Parameters:
Folder Path: Folder path of the folder that will be monitored by Scispot Agent.
Port: Set the port to 2121.
Username: Use admin as the username.
Password: Your password will be the same as your API key used for the Scispot Agent.
Connect to the Agent: Open your FTP client and enter the above parameters. Once connected, you can transfer files directly to the shared local folder monitored by the Scispot Agent.
Data Synchronization: Files transferred via FTP will be automatically synchronized with the Scispot cloud, ensuring that all data from your instruments is centrally managed and accessible.
Use Case
This FTP feature is particularly useful for laboratories and research facilities where instruments like spectrometers, sequencers, or other data-generating devices need to directly transfer their output to the Scispot system. By enabling this, you ensure that data from various sources is consistently and efficiently integrated into your Scispot environment.
Optional for ASTM Protocol Connections: Enabling ASTM Protocol for Instrument Integration
In addition to monitoring file system changes, the Scispot Agent can be configured to facilitate ASTM protocol connections for integrating various instruments that require such connectivity. This feature allows instruments to directly communicate with the Scispot Agent via the ASTM protocol, enabling seamless data transfer and synchronization.
Configuring Your ASTM Client
To establish an ASTM protocol connection with the Scispot Agent, follow these steps:
Set the Connection Parameters:
Check the Enable ASTM Server option.
Port: Set the port.
File Path: Input the file path.
Use Case
This ASTM protocol feature is particularly useful for laboratories and research facilities where instruments like analyzers, diagnostic machines, or other data-generating devices need to directly transfer their output to the Scispot system. By enabling this, you ensure that data from various sources is consistently and efficiently integrated into your Scispot environment.
Optional: Configuring Your Agent for Custom AWS S3 Bucket Data Uploads
Sending Data to Your Custom AWS S3 Bucket
Integrating the Scispot Agent with your custom AWS S3 bucket allows for a seamless and secure transfer of data directly to the cloud, ensuring that your files are stored in a centralized, accessible, and scalable storage solution. Follow these steps to configure the Scispot Agent to send data to your AWS S3 bucket.
Prerequisites
AWS Account: Ensure you have an active AWS account.
S3 Bucket: Create an S3 bucket in your AWS account where the Scispot Agent will upload the data. Note the bucket name for later use.
IAM User: Create an IAM (Identity and Access Management) user with programmatic access and attach policies granting permission to upload files to the specified S3 bucket. Note the access key ID and secret access key.
Configuration Steps
Gather AWS Credentials and Bucket Information:
Access Key ID: Obtained from your IAM user.
Secret Access Key: Obtained from your IAM user.
S3 Bucket Name: The name of your S3 bucket.
AWS Region: The AWS region where your S3 bucket is hosted.
Go to the Source Instrument where your agent is attached:
In the configuration you can add your S3 credentials.
After adding the AWS S3 configuration and enabling the S3 upload feature, Click on update to update the configuration.
Verifying Data Upload
To ensure that your setup is working correctly, create or modify a file in the monitored folder. Check your AWS S3 bucket for the uploaded file to verify that the Scispot Agent is successfully sending data to your S3 bucket.
Ensuring Network Access for Outbound Connections
For the Scispot Agent to function optimally, especially in scenarios where it is required to send data to AWS S3 or communicate with the Scispot cloud, it is crucial to ensure that the agent has access to the internet for outbound connections. This section outlines the necessary steps and considerations for configuring your network to allow the Scispot Agent to establish outbound connections without compromising your network's security.
Network Configuration for Outbound Connections
Outbound Access: The Scispot Agent requires the ability to initiate connections to the internet. This is essential for uploading data to your AWS S3 bucket and communicating with the Scispot cloud services. Ensure that your network firewall or security settings are configured to allow outbound connections from the machine hosting the Scispot Agent.
No Inbound Access Required: The Scispot Agent is designed to only make outbound connections. There is no need to configure your firewall to accept inbound connections from the internet to the agent. This design helps maintain a high level of security by minimizing the agent's exposure to external threats.
Specific Domains and Ports: To further tighten security, you may restrict outbound connections to specific domains and ports that are used by AWS services and the Scispot cloud. For AWS S3, typically, HTTPS access over port 443 is required. Consult AWS documentation for the most current list of endpoints and ports.
Steps to Configure Your Network
Review Firewall Settings: Check your firewall settings to identify any rules that might block outbound connections initiated by the Scispot Agent. Ensure that your firewall is configured to allow these connections.
Configure Outbound Rules: Add outbound firewall rules to permit the Scispot Agent to access the internet. Specify the application or service as the Scispot Agent, and set the action to allow or permit outbound connections.
Limit Access by Domains/Ports: If your security policy requires, limit outbound connections to specific domains (e.g., AWS S3 endpoints) and use the HTTPS protocol over port 443. This approach reduces the risk of data being sent to unauthorized destinations.
Apply and Test Configuration: After configuring your firewall or security settings, apply the changes and test the configuration. You can verify connectivity by checking if the Scispot Agent can successfully upload data to your AWS S3 bucket or communicate with the Scispot cloud.
Security Considerations
Regularly Review Network Policies: Periodically review your network access policies and firewall configurations to ensure they still meet your security and operational requirements.
Monitor Outbound Traffic: Implement monitoring for outbound traffic to detect any unusual patterns or attempts to connect to unauthorized destinations. This can help in identifying potential security issues early.
Running the Agent in Isolated Mode
The Scispot Agent supports an isolated mode for enhanced security and controlled operation. In this mode, the agent operates in a restricted environment, limiting its interaction with external systems and networks. This feature is particularly useful in environments where security and data privacy are of paramount importance.
Prerequisites for Isolated Mode
Before enabling isolated mode, ensure the following requirements are met:
Isolated Environment: The machine running the Scispot Agent in isolated mode should be configured to restrict unnecessary outbound network connections. This setup helps in minimizing potential security risks.
Local System Access: The agent requires access to a local system or network that does not require internet connectivity for its core functions. Ensure that all necessary resources are accessible within this isolated environment.
Configuring the Agent for Isolated Mode
To run the Scispot Agent in isolated mode, you must check for isolation mode in the configuration and update the configuration.
Verifying Isolated Mode Operation
After configuring the agent for isolated mode, execute the script to start the agent. Verify that the agent operates correctly under isolated conditions by observing its behavior and ensuring it does not attempt to make unauthorized external connections.
Troubleshooting
Invalid Serial Number: If you encounter issues related to an invalid serial number, please check that you have entered the serial number correctly and that it matches the one provided for your isolated mode setup.
Connectivity Issues: Ensure that all required local resources are accessible within the isolated environment. The agent may fail to perform as expected if it cannot access necessary local files or services.
By following these steps, you can successfully configure and run the Scispot Agent in isolated mode, providing an additional layer of security and control for your operational environment.
Enabling file removal after processing
To enhance file management and ensure that your local directory remains uncluttered, the Scispot Agent provides an option to automatically remove files after they have been successfully processed and uploaded. By default, this feature is disabled to prevent accidental deletion of files. However, you can easily enable this functionality through the configuration of your agent inside Glue.
Configuring Automatic File Removal:
To run the Scispot Agent in isolated mode, you must check for isolation mode in the configuration and update the configuration.
Considerations
Data Safety: Before enabling this feature, ensure that you have a reliable backup system in place or that the files are no longer needed. Once a file is deleted, it cannot be recovered from the local system.
Verification: It's recommended to manually verify that the files are correctly uploaded to their destination before enabling automatic removal, especially if this feature is being used for the first time.
By following these steps, you can configure the Scispot Agent to automatically remove processed files from your local directory, helping to maintain a clean and efficient file management system.