Files
Datagrok lets you work with files and directories on your system from the convenience of a web browser. You can browse, preview, open, create, delete, rename, download, clone, and share files and directories. When you sign up for Datagrok, a personal directory called Home is automatically created for you. Additionally, you can connect to popular file systems, including the Amazon S3 bucket, Dropbox, Google Drive, and Git, as well as Windows and Linux network shares.
Connecting to an SMB file storage is only available for on-premise deployment and is not available on the public Datagrok instance (public.datagrok.ai).
You can create custom connectors and read files programmatically.
Connecting to file storage
To connect to your file storage, follow these steps:
- Go to Data > Files.
- Open the New file share dialog (Toolbox > Actions > New file share). Alternatively, click the New file share icon on the Menu Riboon.
- In the dialog, choose the data source from the Data Source dropdown. The dialog updates with connection-specific parameters.
- Set the parameters.
- Click TEST to test the connection, then click OK to save it.
Some connection parameters have unique characteristics, and it's important to specify them correctly:
Directory path. When connecting to the root directory, leave the Dir field empty. Otherwise, enter a directory path.
Credentials. You can specify credentials manually or using the Secrets Manager, such as the AWS Secrets Manager. When entered manually, Datagrok stores secrets in a secure privilege management system. To specify who can change the connection credentials, click the Gear icon and select from the Credential owner dropdown.
:::caution
When connecting to public buckets in AWS S3, always check the Anonymous checkbox.
:::
Once you have established a connection to a folder in your file system, the folder appears in the File Manager under the corresponding data source. This connection is referred to as a file share. You can view the files and subfolders within the file share by expanding it.
Like other objects in Datagrok, newly created connections are only visible to the user who created them. To let others access the file share, you must share it (right-click the connection and select Share... from the list of options).
To modify a connection, right-click it and select Edit... from the list of options. To quickly create a connection similar to an existing one, right-click it and select Clone...
Importing text
Datagrok supports text-to-tabular-data conversion for delimiter-separated files with an option to manually edit or customize data during import. To use this feature, open the Text Manager (Data
Text) and load a file or paste text into the Text Editor area. Here you can change the data as needed. Adjust default import parameters like delimiters, decimal separators, and header settings in the Toolbox on the left. By default, changes are automatically applied and displayed in the Preview, which updates as you modify the data. To manually sync edits, disable the Auto Sync checkbox and use the Sync button.
When satisfied with the data, click Done to open the dataframe in Datagrok. From there, you can export it, make further edits, save it as a project, or share with others via URL.
File Manager
The File Manager is an interface that allows you to manage connections, browse and preview file content, and perform standard file and folder actions such as opening, downloading, deleting, and renaming. To access an object's context actions, right-click it or left-click and expand the Actions pane in the Context Panel on the left. By clicking a file or folder in the File Manager, you can open its preview. Double-clicking a file opens it in Datagrok, and double-clicking a folder expands its content.
If you don't see a certain action, it may be due to insufficient permissions. For files and folders shared with you, contact the credentials owner. If you are a credentials owner, contact the data source owner.
In addition to the hierarchical browsing, the File Manager offers advanced preview and data augmentation capabilities using Directory, Preview, and Context Panel.
The Directory section shows the contents of your current folder. Click a file to see its preview and properties, or right-click it for more actions. Use the search bar to search for files and folders within your current directory. The search bar allows you to search for items by name, file extension, or metadata.
For folders, the Preview generates a treemap that highlights the largest items. For files, the functionality varies based on the file's format and data properties. It includes custom viewers for supported formats, such as interactive spreadsheets for displaying tabular data, cell and image renderers, and chemical and biological structure viewers. You can also view the content of ZIP files and edit Markdown, TXT, and HTML files.
File preview is limited to files under 10MB. The platform won't display larger files. Unsupported file formats cannot be previewed, but you can download them.
You can add custom formats using package extensions. In addition, you can create organization-specific previews:
Example: Create custom file viewers
In this example, a script is executed against the folder content. If the folder contains files that match the file extension parameter PDB, the Preview displays a custom NGL viewer to visualize the molecule.
To add a custom viewer, you have two options:
- Develop in JavaScript using the Datagrok JavaScript API.
- Use the visualizations available for popular programming languages like Python, R, or Julia.
To learn more about each option, see Develop custom viewer.
Example: Create custom folder viewers
The Context Panel provides additional information about a selected file or folder, and the ability to execute conext actions. For example, when you click a CSV file, the Context Panel updates to show the file's metadata, available context actions, and other relevant information. If you subsequently click any of the dataframe's columns in the Preview, the Context Panel will update to display information and actions specific to that column, such as summary statistics for the column under Stats, or its data and semantic types under Details.
Context Panel can be extended. You can add custom info panes and context actions.
Example: Image augmentation
In this example, a Python script creates a custom info pane called Cell Imaging Segmentation. This script executes against JPEG and JPG files during the indexing process and extracts custom metadata (such as the number of cells) and performs predefined transformations (such as cell segmentation). When a user selects the corresponding image, the Context Panel shows a custom info panel that displays the augmented file preview and the number of detected cell segments.
File sharing and access control
You can share files in two ways: by sharing the actual file (or folder), or by sharing an URL that points to it. To share an URL, open the file in Datagrok and copy the URL from the address bar. To access the file from the link provided, users must have permissions to open it. Once the file is open, users can download the file and then upload it to their Home directory, or save the file as a project. The URL links never expire and can't be revoked.
For tabular formats, you can create dynamic dashboards and share them with others via URL or reference them on external websites. To learn more about dynamic data updates, see Dynamic data.
Another option is to share directly in Datagrok by creating a file share and specifying access privileges for each shared item, such as separate files and subfolders. Once the item is shared, it appears in the recipient's File Manager. When using this method, you have several options:
- Share a connection (root folder) to give access to the entire directory.
- Share a folder to give access to the content of individual folders in your directory.
To share, follow these steps:
Right-click the item you want to share and select Share... from its context menu. The Share... dialog opens.
In the identity/email field, start typing a person's name, username, email, or group name, and pick from the list of matching identities.
From the respective dropdowns, select access privileges for either or both: (1) the connection and (2) individual files/folders. You can select any or all of the following options:
Can view: Users can view, open, and download
Can edit: Users can rename, edit, delete, and reshare with any other user or group.
cautionA file's name and namespace are encoded within the URL. When you rename a file (or its location), the link changes accordingly, which may cause broken URL links, script errors, and similar issues.
Optional. Enter a description in the text field provided. You may also notify the users you share with. If you don't want to send a notification, clear the Send notification checkbox.
noteTo send an email notification, enter the user's email in the identity/email field. The email notification contains a link to the shared item and entered description. If you enter a user or group name, they will be notified via the Datagrok interface.
Click OK to share. Once shared, the shared item appears in the recipient's File Manager.
To inspect or quickly adjust access permissions to your file shares, send comments to those you're sharing with, and more, use the Sharing info pane in the Context Panel.
Resources
