Efficiently Downloading Large Files from Google Drive Using Command-Line Tools

Introduction

Downloading large files directly from Google Drive via command-line tools can be challenging due to the "virus scan" prompt that appears for files over a certain size. This tutorial will guide you through various methods and tools to bypass this limitation, enabling seamless downloading of large files using wget, curl, Python scripts, and third-party command-line utilities.

Understanding Google Drive’s Download Mechanism

Google Drive restricts the direct download of large files via simple HTTP requests due to security measures. When attempting to download large files using standard methods like wget or curl, you might encounter a web page asking for confirmation past a virus scan. This is where more advanced techniques and tools come into play.

Tools Overview

1. gdown: A Python-based tool specifically designed for downloading files from Google Drive.
2. Python Requests Module: A versatile module to handle complex HTTP requests, including handling additional tokens required by Google Drive.
3. gdrive: A cross-platform command-line tool capable of downloading both files and folders from Google Drive.

Using gdown for Downloading Files

Installation

To use gdown, you need Python installed on your system along with the package itself:

pip install gdown

Usage

1. Get File ID: Right-click the file in Google Drive and select "Get link". Extract the file ID from the URL.

2. Download the File:

   For files:

   gdown https://drive.google.com/uc?id=<file_id>
   

   For folders:

   gdown --folder https://drive.google.com/drive/folders/<file_id>
   

Example

To download a file with ID 0B7EVK8r0v71pOXBhSUdJWU1MYUk:

gdown https://drive.google.com/uc?id=0B7EVK8r0v71pOXBhSUdJWU1MYUk

Caveats

• Works only with files accessible to anyone with the link.
• Cannot download more than 50 files into a single folder unless they are zipped.

Using Python Requests Module for Advanced Downloads

For those comfortable with scripting, using Python’s requests library can provide more control over downloading large files from Google Drive.

Script Example

import requests

def download_file_from_google_drive(id, destination):
    def get_confirm_token(response):
        for key, value in response.cookies.items():
            if key.startswith('download_warning'):
                return value
        return None

    session = requests.Session()
    response = session.get(f'https://drive.google.com/uc?export=download&id={id}', stream=True)
    
    token = get_confirm_token(response)

    if token:
        params = {'confirm': token}
        response = session.get(f'https://drive.google.com/uc?export=download&id={id}', params=params, stream=True)

    with open(destination, "wb") as f:
        for chunk in response.iter_content(1024):
            if chunk:  # filter out keep-alive new chunks
                f.write(chunk)

# Usage
file_id = 'your-file-id'
destination_path = '/path/to/destination/file.ext'
download_file_from_google_drive(file_id, destination_path)

This script handles the confirmation token required for large files and downloads them directly.

Using gdrive for Downloading Files and Folders

Installation

Download the appropriate binary from gdrive’s GitHub releases for your OS:

gunzip gdrive_2.1.1_linux_amd64.tar.gz
sudo mkdir /usr/local/bin/gdrive
sudo cp gdrive-linux-amd64 /usr/local/bin/gdrive
sudo chmod a+x /usr/local/bin/gdrive

Usage

1. Get File ID: Right-click the file in Google Drive and select "Get Link".

2. Download the File:

   gdrive download FILE_ID
   

3. Authentication: On first run, authenticate via a browser link provided by gdrive.

Rate Limiting

To control the download speed (useful for network management):

gdrive download --stdout FILE_ID | pv -br -L 90k > file.ext

Conclusion

By using these methods and tools, you can efficiently download large files from Google Drive without manual intervention. Each tool offers different features: gdown is simple and effective for basic downloads, Python requests provide flexibility and control, while gdrive supports complex scenarios including folder downloads and non-public file access.