Uploading to Arweave

Overview

While AR.IO provides powerful tools for accessing and interacting with data on Arweave, that data must first be uploaded to the network. This guide will walk you through the process of uploading data to Arweave using the Turbo SDK, which provides a streamlined experience for data uploads.

Installing Turbo SDK

# For Node.js
npm install @ardrive/turbo-sdk

# For Yarn users
yarn add @ardrive/turbo-sdk

Authentication

Node.js Environment

Node.js Authentication

import { TurboFactory } from '@ardrive/turbo-sdk'
import fs from 'fs'

// Load your Arweave JWK file
const jwk = JSON.parse(fs.readFileSync('wallet.json', 'utf-8'))

const turbo = await TurboFactory.authenticated({
  privateKey: jwk, // ArweaveJWK type
  token: 'arweave', // Default token type
})

Browser Environment

Browser Auth Examples

import { TurboFactory, ArConnectSigner } from '@ardrive/turbo-sdk/web'

async function initializeTurbo() {
  await window.arweaveWallet.connect([
    'ACCESS_ADDRESS',
    'ACCESS_PUBLIC_KEY',
    'SIGN_TRANSACTIONS',
    'SIGN_MESSAGE',
    'SIGNATURE',
  ])

  const turbo = await TurboFactory.authenticated({
    signer: new ArConnectSigner(window.arweaveWallet),
  })
}

Purchasing Turbo Credits

Turbo Credits are the payment medium used by the Turbo Upload Service. Each Credit represents a 1:1 conversion from the upload power of the Arweave native token (AR). Turbo Credits can be purchased with fiat currency via the Turbo Top Up App, or with supported cryptocurrencies via the Turbo SDK. Learn more about Turbo Credits and available methods for purchasing them here.

Node.js Environment

Purchasing Credits With Crypto: Node.js

import { TurboFactory, WinstonToTokenAmount } from '@ardrive/turbo-sdk'

// Initialize authenticated client
const turbo = await TurboFactory.authenticated({ privateKey: jwk })

// Top up with AR tokens
const topUpResult = await turbo.topUpWithTokens({
  tokenAmount: WinstonToTokenAmount(100_000_000), // 0.0001 AR
})

Browser Environment

In a browser environment, the topUpWithTokens method is not available. Instead, you'll need to manually send tokens to the Turbo wallet address and then submit the transaction for processing. Here are detailed examples for each supported chain:

Browser Top-Up Examples

import { TurboFactory } from '@ardrive/turbo-sdk/web'
import Arweave from 'arweave'
import axios from 'axios'

const TURBO_AR_ADDRESS = 'JNC6vBhjHY1EPwV3pEeNmrsgFMxH5d38_LHsZ7jful8'
const AR_AMOUNT = 0.0001 // Amount in AR

// Function to send AR and wait for confirmation
const sendArToTurbo = async () => {
  if (!window.arweaveWallet) {
    throw new Error('Please install Wander')
  }

  // Initialize Arweave
  const arweave = Arweave.init({
    host: 'arweave.net',
    port: 443,
    protocol: 'https',
  })

  // Create transaction
  const transaction = await arweave.createTransaction({
    target: TURBO_AR_ADDRESS,
    quantity: arweave.ar.arToWinston(AR_AMOUNT.toString()),
  })

  // Sign and post transaction
  await window.arweaveWallet.sign(transaction)
  const response = await arweave.transactions.post(transaction)

  return transaction.id
}

// Function to submit transaction with retries
const submitTransactionWithRetries = async (txId, maxRetries = 3) => {
  let retries = 0
  while (retries < maxRetries) {
    try {
      const response = await turbo.submitFundTransaction({ txId })
      return response
    } catch (error) {
      retries++
      if (retries === maxRetries) throw error
      // Wait 30 seconds before retrying
      await new Promise((resolve) => setTimeout(resolve, 30000))
    }
  }
}

// Complete top-up process
const topUpWithAr = async () => {
  try {
    // Send AR and get transaction ID
    const txId = await sendArToTurbo()
    console.log('Transaction sent:', txId)

    // Wait 36 minutes for chain settlement
    await new Promise((resolve) => setTimeout(resolve, 36 * 60 * 1000))

    // Submit transaction with retries
    const response = await submitTransactionWithRetries(txId)
    console.log('Credits added:', response)
  } catch (error) {
    console.error('Top-up failed:', error)
  }
}

Note: The wait times for chain settlement are approximate and may need adjustment based on network conditions:

Ethereum: ~15 minutes

Solana: ~400-600 milliseconds

Arweave: ~30-36 minutes

Polygon: ~2-3 seconds

Base: ~2-5 seconds

KYVE: ~5 minutes

Uploading Files and Folders

Once you have purchased Turbo credits, you can upload files and folders to Arweave. The process is the same regardless of which token type you used for authentication, but differs between Node.js and browser environments.

Node.js Environment

Node.js Upload Examples

import { TurboFactory } from '@ardrive/turbo-sdk'
import fs from 'fs'
import path from 'path'
import mime from 'mime-types'

// Initialize authenticated client
const turbo = await TurboFactory.authenticated({ privateKey: jwk })

// Function to upload a single file
const uploadFile = async (filePath) => {
  try {
    // Get file info
    const fileSize = fs.statSync(filePath).size
    const mimeType = mime.lookup(filePath) || 'application/octet-stream'

    // Upload file
    const result = await turbo.uploadFile({
      fileStreamFactory: () => fs.createReadStream(filePath),
      fileSizeFactory: () => fileSize,
      dataItemOpts: {
        tags: [
          {
            name: 'Content-Type',
            value: mimeType,
          },
        ],
      },
    })

    console.log('File uploaded!', {
      id: result.id,
      url: `https://arweave.net/${result.id}`,
      owner: result.owner,
      dataCaches: result.dataCaches,
    })

    return result
  } catch (error) {
    console.error('Upload failed:', error)
    throw error
  }
}

// Example usage
await uploadFile('./path/to/your/file.pdf')

Browser Environment

Browser Upload Examples

import { TurboFactory } from '@ardrive/turbo-sdk/web'

// Initialize authenticated client
const turbo = await TurboFactory.authenticated({ signer })

// HTML input element
<input
  type="file"
  id="file-input"
  accept="image/*,video/*,audio/*,.pdf,.txt"
/>

// Function to upload a single file
const uploadFile = async (file) => {
  try {
    const result = await turbo.uploadFile({
      fileStreamFactory: () => file.stream(),
      fileSizeFactory: () => file.size,
      dataItemOpts: {
        tags: [
          {
            name: 'Content-Type',
            value: file.type || 'application/octet-stream'
          }
        ]
      }
    })

    console.log('File uploaded!', {
      id: result.id,
      url: `https://arweave.net/${result.id}`,
      owner: result.owner,
      dataCaches: result.dataCaches
    })

    return result
  } catch (error) {
    console.error('Upload failed:', error)
    throw error
  }
}

// Example usage with file input
const fileInput = document.getElementById('file-input')
fileInput.addEventListener('change', async (event) => {
  const file = fileInput.files[0]
  if (!file) return
  await uploadFile(file)
})

// Example usage with drag and drop
const dropZone = document.getElementById('drop-zone')
dropZone.addEventListener('dragover', (e) => {
  e.preventDefault()
  e.stopPropagation()
  dropZone.classList.add('drag-over')
})

dropZone.addEventListener('dragleave', (e) => {
  e.preventDefault()
  e.stopPropagation()
  dropZone.classList.remove('drag-over')
})

dropZone.addEventListener('drop', async (e) => {
  e.preventDefault()
  e.stopPropagation()

  const file = e.dataTransfer.files[0]
  if (!file) return

  await uploadFile(file)
})

Important Notes:

For single file uploads, always include a Content-Type tag to ensure proper file viewing

The fileStreamFactory must return a NEW stream each time it's called

Folder uploads automatically detect and set Content-Type tags for all files

You can specify additional tags in dataItemOpts for both file and folder uploads

The maxConcurrentUploads option controls how many files are uploaded simultaneously

Use throwOnFailure: true to ensure all files are uploaded successfully

Complete Examples

Here are complete examples showing how to authenticate, check balances, and handle lazy funding for uploads. These examples demonstrate the full workflow from start to finish.

Node.js Environment

Complete Node.js Example

import { TurboFactory } from '@ardrive/turbo-sdk'
import fs from 'fs'
import path from 'path'
import mime from 'mime-types'
import axios from 'axios'

// Constants
const FREE_UPLOAD_SIZE = 100 * 1024 // 100KB in bytes
const PRICE_BUFFER = 1.1 // 10% buffer for price fluctuations

// Initialize authenticated client
const turbo = await TurboFactory.authenticated({ privateKey: jwk })

// Function to get token price from CoinGecko
const getTokenPrice = async (token: string) => {
  const response = await axios.get(
    `https://api.coingecko.com/api/v3/simple/price?ids=${token}&vs_currencies=usd`,
  )
  return response.data[token].usd
}

// Function to calculate required token amount
const calculateTokenAmount = async (wincAmount: string, tokenType: string) => {
  // Get fiat rates for 1 GiB
  const fiatRates = await turbo.getFiatRates()
  const usdPerGiB = fiatRates.usd

  // Convert winc to GiB
  const wincPerGiB = 1_000_000_000_000 // 1 GiB in winc
  const requiredGiB = Number(wincAmount) / wincPerGiB
  const requiredUsd = requiredGiB * usdPerGiB

  // Get token price
  const tokenPrice = await getTokenPrice(tokenType)
  const tokenAmount = (requiredUsd / tokenPrice) * PRICE_BUFFER

  return tokenAmount
}

// Function to check balance and fund if needed
const ensureSufficientBalance = async (fileSize: number, tokenType: string) => {
  // Check current balance
  const balance = await turbo.getBalance()
  const currentWinc = BigInt(balance.controlledWinc)

  // If file is under 100KB, it's free
  if (fileSize <= FREE_UPLOAD_SIZE) {
    return true
  }

  // Get upload cost
  const costs = await turbo.getUploadCosts({ bytes: [fileSize] })
  const requiredWinc = BigInt(costs[0].winc)

  // If we have enough balance, return true
  if (currentWinc >= requiredWinc) {
    return true
  }

  // Calculate and purchase required tokens
  const tokenAmount = await calculateTokenAmount(
    requiredWinc.toString(),
    tokenType,
  )

  // Top up with tokens
  await turbo.topUpWithTokens({
    tokenAmount: tokenAmount,
  })

  return true
}

// Function to upload a file
const uploadFile = async (filePath: string) => {
  try {
    // Get file info
    const fileSize = fs.statSync(filePath).size
    const mimeType = mime.lookup(filePath) || 'application/octet-stream'

    // Ensure sufficient balance
    await ensureSufficientBalance(fileSize, 'arweave')

    // Upload file
    const result = await turbo.uploadFile({
      fileStreamFactory: () => fs.createReadStream(filePath),
      fileSizeFactory: () => fileSize,
      dataItemOpts: {
        tags: [
          {
            name: 'Content-Type',
            value: mimeType,
          },
        ],
      },
    })

    console.log('File uploaded!', {
      id: result.id,
      url: `https://arweave.net/${result.id}`,
      owner: result.owner,
      dataCaches: result.dataCaches,
    })

    return result
  } catch (error) {
    console.error('Upload failed:', error)
    throw error
  }
}

// Example usage
await uploadFile('./path/to/your/file.pdf')

Browser Environment

Complete Browser Example

import { TurboFactory } from '@ardrive/turbo-sdk/web'
import Arweave from 'arweave'
import axios from 'axios'

// Constants
const FREE_UPLOAD_SIZE = 100 * 1024 // 100KB in bytes
const PRICE_BUFFER = 1.1 // 10% buffer for price fluctuations
const TURBO_AR_ADDRESS = 'JNC6vBhjHY1EPwV3pEeNmrsgFMxH5d38_LHsZ7jful8'

// Initialize authenticated client with Wander
if (!window.arweaveWallet) {
  throw new Error('Please install Wander')
}

// Initialize Arweave
const arweave = Arweave.init({
  host: 'arweave.net',
  port: 443,
  protocol: 'https'
})

const turbo = await TurboFactory.authenticated({
  privateKey: window.arweaveWallet,
  token: 'arweave'
})

// Function to get token price from CoinGecko
const getTokenPrice = async (token: string) => {
  const response = await axios.get(
    `https://api.coingecko.com/api/v3/simple/price?ids=${token}&vs_currencies=usd`
  )
  return response.data[token].usd
}

// Function to calculate required token amount
const calculateTokenAmount = async (
  wincAmount: string,
  tokenType: string
) => {
  // Get fiat rates for 1 GiB
  const fiatRates = await turbo.getFiatRates()
  const usdPerGiB = fiatRates.usd

  // Get winc cost for 1 GiB
  const costs = await turbo.getUploadCosts({ bytes: [1024 * 1024 * 1024] }) // 1 GiB in bytes
  const wincPerGiB = BigInt(costs[0].winc)

  // Calculate cost per winc in USD
  const usdPerWinc = Number(usdPerGiB) / Number(wincPerGiB)

  // Calculate required USD amount
  const requiredUsd = Number(wincAmount) * usdPerWinc

  // Get token price
  const tokenPrice = await getTokenPrice(tokenType)
  const tokenAmount = (requiredUsd / tokenPrice) * PRICE_BUFFER

  return tokenAmount
}

// Function to check balance and fund if needed
const ensureSufficientBalance = async (
  fileSize: number,
  tokenType: string
) => {
  // Check current balance
  const balance = await turbo.getBalance()
  const currentWinc = BigInt(balance.controlledWinc)

  // If file is under 100KB, it's free
  if (fileSize <= FREE_UPLOAD_SIZE) {
    return true
  }

  // Get upload cost
  const costs = await turbo.getUploadCosts({ bytes: [fileSize] })
  const requiredWinc = BigInt(costs[0].winc)

  // If we have enough balance, return true
  if (currentWinc >= requiredWinc) {
    return true
  }

  // Calculate and purchase required tokens
  const tokenAmount = await calculateTokenAmount(
    requiredWinc.toString(),
    tokenType
  )

  // Create transaction
  const transaction = await arweave.createTransaction({
    target: TURBO_AR_ADDRESS,
    quantity: arweave.ar.arToWinston(tokenAmount.toString())
  })

  // Sign and post transaction
  await window.arweaveWallet.sign(transaction)
  await arweave.transactions.post(transaction)

  // Wait for confirmation (typically 30-36 minutes)
  await new Promise((resolve) => setTimeout(resolve, 36 * 60 * 1000))

  // Submit transaction to Turbo
  await turbo.submitFundTransaction({
    txId: transaction.id
  })

  return true
}

// Function to upload a file
const uploadFile = async (file: File) => {
  try {
    // Ensure sufficient balance
    await ensureSufficientBalance(file.size, 'arweave')

    // Upload file
    const result = await turbo.uploadFile({
      fileStreamFactory: () => file.stream(),
      fileSizeFactory: () => file.size,
      dataItemOpts: {
        tags: [
          {
            name: 'Content-Type',
            value: file.type || 'application/octet-stream'
          }
        ]
      }
    })

    console.log('File uploaded!', {
      id: result.id,
      url: `https://arweave.net/${result.id}`,
      owner: result.owner,
      dataCaches: result.dataCaches
    })

    return result
  } catch (error) {
    console.error('Upload failed:', error)
    throw error
  }
}

// HTML input element
<input
  type="file"
  id="file-input"
  accept="image/*,video/*,audio/*,.pdf,.txt"
/>

// Example usage with file input
const fileInput = document.getElementById('file-input')
fileInput.addEventListener('change', async (event) => {
  const file = fileInput.files[0]
  if (!file) return
  await uploadFile(file)
})

// Example usage with drag and drop
const dropZone = document.getElementById('drop-zone')
dropZone.addEventListener('dragover', (e) => {
  e.preventDefault()
  e.stopPropagation()
  dropZone.classList.add('drag-over')
})

dropZone.addEventListener('dragleave', (e) => {
  e.preventDefault()
  e.stopPropagation()
  dropZone.classList.remove('drag-over')
})

dropZone.addEventListener('drop', async (e) => {
  e.preventDefault()
  e.stopPropagation()
  dropZone.classList.remove('drag-over')

  const file = e.dataTransfer.files[0]
  if (!file) return

  await uploadFile(file)
})