8000 minio-py/docs/API.md at patch-1 · poornas/minio-py · GitHub
[go: up one dir, main page]
Skip to content

API.md

Latest commit

 

History

History
1040 lines (792 loc) · 32.5 KB

API.md

File metadata and controls

1040 lines (792 loc) · 32.5 KB
Raw

Python Client API Reference Slack

Initialize Minio Client object.

Minio

from minio import Minio
from minio.error import ResponseError

minioClient = Minio('play.minio.io:9000',
                  access_key='Q3AM3UQ867SPQQA43P2F',
                  secret_key='zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG',
                  secure=True)

AWS S3

from minio import Minio
from minio.error import ResponseError

s3Client = Minio('s3.amazonaws.com',
                 access_key='YOUR-ACCESSKEYID',
                 secret_key='YOUR-SECRETACCESSKEY',
                 secure=True)
Bucket operations Object operations Presigned operations Bucket policy/notification operations
make_bucket get_object presigned_get_object get_bucket_policy
list_buckets put_object presigned_put_object set_bucket_policy
bucket_exists copy_object presigned_post_policy get_bucket_notification
remove_bucket stat_object set_bucket_notification
list_objects remove_object remove_all_bucket_notification
list_objects_v2 remove_objects listen_bucket_notification
list_incomplete_uploads remove_incomplete_upload
fput_object
fget_object
get_partial_object

1. Constructor

Minio(endpoint, access_key=None, secret_key=None, secure=True)
Minio(endpoint, access_key=None, secret_key=None, secure=True, region=None, timeout=None)
Initializes a new client object.

Parameters

Param Type Description
endpoint string S3 object storage endpoint.
access_key string Access key for the object storage endpoint. (Optional if you need anonymous access).
secret_key string Secret key for the object storage endpoint. (Optional if you need anonymous access).
secure bool Set this value to True to enable secure (HTTPS) access. (Optional defaults to True).
region string Set this value to override automatic bucket location discovery. (Optional defaults to None).
timeout float Set this value to control how long requests are allowed to run before being aborted. (Optional defaults to None)

Example

Minio

from minio import Minio
from minio.error import ResponseError

minioClient = Minio('play.minio.io:9000',
                    access_key='Q3AM3UQ867SPQQA43P2F',
                    secret_key='zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG')

AWS S3

from minio import Minio
from minio.error import ResponseError

s3Client = Minio('s3.amazonaws.com',
                 access_key='ACCESS_KEY',
                 secret_key='SECRET_KEY')

2. Bucket operations

make_bucket(bucket_name, location='us-east-1')

Creates a new bucket.

Parameters

Param Type Description
bucket_name string Name of the bucket.
location string Default value is us-east-1 Region where the bucket is created. Valid values are listed below:
us-east-1
us-west-1
us-west-2
eu-west-1
eu-central-1
ap-southeast-1
ap-northeast-1
ap-southeast-2
sa-east-1
cn-north-1

Example

try:
    minioClient.make_bucket("mybucket", location="us-east-1")
except ResponseError as err:
    print(err)

list_buckets()

Lists all buckets.

Parameters

Return Type Description
bucketList function List of all buckets.
bucket.name string Bucket name.
bucket.creation_date time Time: date when bucket was created.

Example

buckets = minioClient.list_buckets()
for bucket in buckets:
    print(bucket.name, bucket.creation_date)

bucket_exists(bucket_name)

Checks if a bucket exists.

Parameters

Param Type Description
bucket_name string Name of the bucket.

Example

try:
    print(minioClient.bucket_exists("mybucket"))
except ResponseError as err:
    print(err)

remove_bucket(bucket_name)

Removes a bucket.

Parameters

Param Type Description
bucket_name string Name of the bucket.

Example

try:
    minioClient.remove_bucket("mybucket")
except ResponseError as err:
    print(err)

list_objects(bucket_name, prefix, recursive=False)

Lists objects in a bucket.

Parameters

Param Type Description
bucket_name string Name of the bucket.
objectPrefix string The prefix of the objects that should be listed.
recursive bool True indicates recursive style listing and False indicates directory style listing delimited by '/'. Optional default is False.

Return Value

Param Type Description
object Object Iterator for all the objects in the bucket, the object is of the format listed below:
Param Type Description
object.object_name string name of the object.
object.size int size of the object.
object.etag string etag of the object.
object.last_modified datetime.datetime modified time stamp.

Example

# List all object paths in bucket that begin with my-prefixname.
objects = minioClient.list_objects('mybucket', prefix='my-prefixname',
                              recursive=True)
for obj in objects:
    print(obj.bucket_name, obj.object_name.encode('utf-8'), obj.last_modified,
          obj.etag, obj.size, obj.content_type)

list_objects_v2(bucket_name, prefix, recursive=False)

Lists objects in a bucket using the Version 2 API.

Parameters

Param Type Description
bucket_name string Name of the bucket.
objectPrefix string The prefix of the objects that should be listed.
recursive bool True indicates recursive style listing and False indicates directory style listing delimited by '/'. Optional default is False.

Return Value

Param Type Description
object Object Iterator for all the objects in the bucket, the object is of the format listed below:
Param Type Description
object.object_name string name of the object.
object.size int size of the object.
object.etag string etag of the object.
object.last_modified datetime.datetime modified time stamp.

Example

# List all object paths in bucket that begin with my-prefixname.
objects = minioClient.list_objects_v2('mybucket', prefix='my-prefixname',
                              recursive=True)
for obj in objects:
    print(obj.bucket_name, obj.object_name.encode('utf-8'), obj.last_modified,
          obj.etag, obj.size, obj.content_type)

list_incomplete_uploads(bucket_name, prefix, recursive=False)

Lists partially uploaded objects in a bucket.

Parameters

Param Type Description
bucket_name string Name of the bucket.
prefix string The prefix of the incomplete objects uploaded should be listed.
recursive bool True indicates recursive style listing and False indicates directory style listing delimited by '/'. Optional default is False.

Return Value

Param Type Description
multipart_obj Object Iterator of multipart objects of the format described below:
Param Type Description
multipart_obj.object_name string name of the incomplete object.
multipart_obj.upload_id string upload ID of the incomplete object.
multipart_obj.size int size of the incompletely uploaded object.

Example

# List all object paths in bucket that begin with my-prefixname.
uploads = minioClient.list_incomplete_uploads('mybucket',
                                         prefix='my-prefixname',
                                         recursive=True)
for obj in uploads:
    print(obj.bucket_name, obj.object_name, obj.upload_id, obj.size)

get_bucket_policy(bucket_name, prefix)

Gets current policy of a bucket.

Parameters

Param Type Description
bucket_name string Name of the bucket.
prefix string The prefix of objects to get current policy.

Return Value

Param Type Description
Policy minio.policy.Policy Policy enum. Policy.READ_ONLY,Policy.WRITE_ONLY,Policy.READ_WRITE or Policy.NONE.

Example

# Get current policy of all object paths in bucket that begin with my-prefixname.
policy = minioClient.get_bucket_policy('mybucket',
                                       'my-prefixname')
print(policy)

set_bucket_policy(bucket_name, prefix, policy)

Set a bucket policy for a specified bucket. If prefix is not empty, the bucket policy will only be assigned to objects that fit the given prefix.

Parameters

Param Type Description
bucket_name string Name of the bucket.
prefix string The prefix of objects to get current policy.
Policy minio.policy.Policy Policy enum. Policy.READ_ONLY,Policy.WRITE_ONLY,Policy.READ_WRITE or Policy.NONE.

Example

# Set policy Policy.READ_ONLY to all object paths in bucket that begin with my-prefixname.
minioClient.set_bucket_policy('mybucket',
                              'my-prefixname',
                              Policy.READ_ONLY)

get_bucket_notification(bucket_name)

Fetch the notifications configuration on a bucket.

Parameters

Param Type Description
bucket_name string Name of the bucket.

Return Value

Param Type Description
notification dict If there is no notification configuration, an empty dictionary is returned. Otherwise it has the same structure as the argument to set_bucket_notification

Example

# Get the notifications configuration for a bucket.
notification = minioClient.get_bucket_notification('mybucket')
# If no notification is present on the bucket:
# notification == {}

set_bucket_notification(bucket_name, notification)

Set notification configuration on a bucket.

Parameters

Param Type Description
bucket_name string Name of the bucket.
notification dict Non-empty dictionary with the structure specified below.

The notification argument has the following structure:

  • (dict) --
    • TopicConfigurations (list) -- Optional list of service configuration items specifying AWS SNS Topics as the target of the notification.
    • QueueConfigurations (list) -- Optional list of service configuration items specifying AWS SQS Queues as the target of the notification.
    • CloudFunctionconfigurations (list) -- Optional list of service configuration items specifying AWS Lambda Cloud functions as the target of the notification.

At least one of the above items needs to be specified in the notification argument.

The "service configuration item" alluded to above has the following structure:

  • (dict) --
    • Id (string) -- Optional Id for the configuration item. If not specified, it is auto-generated by the server.
    • Arn (string) -- Specifies the particular Topic/Queue/Cloud Function identifier.
    • Events (list) -- A non-empty list of event-type strings from: 's3:ReducedRedundancyLostObject', 's3:ObjectCreated:*', 's3:ObjectCreated:Put', 's3:ObjectCreated:Post', 's3:ObjectCreated:Copy', 's3:ObjectCreated:CompleteMultipartUpload', 's3:ObjectRemoved:*', 's3:ObjectRemoved:Delete', 's3:ObjectRemoved:DeleteMarkerCreated'
    • Filter (dict) -- An optional dictionary container of object key name based filter rules.
      • Key (dict) -- Dictionary container of object key name prefix and suffix filtering rules.
        • FilterRules (list) -- A list of containers that specify the criteria for the filter rule.
          • (dict) -- A dictionary container of key value pairs that specify a single filter rule.
            • Name (string) -- Object key name with value 'prefix' or 'suffix'.
            • Value (string) -- Specify the value of the prefix/suffix to which the rule applies.

There is no return value. If there are errors from the target server/service, a ResponseError is thrown. If there are validation errors, InvalidArgumentError or TypeError may be thrown. The input configuration cannot be empty - to delete the notification configuration on a bucket, use the remove_all_bucket_notifications() API.

Example

notification = {
    'QueueConfigurations': [
        {
            'Id': '1',
            'Arn': 'arn1',
            'Events': ['s3:ObjectCreated:*'],
            'Filter': {
                'Key': {
                    'FilterRules': [
                        {
                            'Name': 'prefix',
                            'Value': 'abc'
                        }
                    ]
                }
            }
        }
    ],
    'TopicConfigurations': [
        {
            'Arn': 'arn2',
            'Events': ['s3:ObjectCreated:*'],
            'Filter': {
                'Key': {
                    'FilterRules': [
                        {
                            'Name': 'suffix',
                            'Value': '.jpg'
                        }
                    ]
                }
            }
        }
    ],
    'CloudFunctionConfigurations': [
        {
            'Arn': 'arn3',
            'Events': ['s3:ObjectRemoved:*'],
            'Filter': {
                'Key': {
                    'FilterRules': [
                        {
                            'Name': 'suffix',
                            'Value': '.jpg'
                        }
                    ]
                }
            }
        }
    ]
}


try:
    minioClient.set_bucket_notification('mybucket', notification)
except ResponseError as err:
    # handle error response from service.
    print(err)
except (ArgumentError, TypeError) as err:
    # should happen only during development. Fix the notification argument
    print(err)

remove_all_bucket_notifications(bucket_name)

Remove all notifications configured on the bucket.

Parameters

Param Type Description
bucket_name string Name of the bucket.

There is no returned value. A ResponseError exception is thrown if the operation did not complete successfully.

Example

# Remove all the notifications config for a bucket.
minioClient.remove_all_bucket_notifications('mybucket')

listen_bucket_notification(bucket_name, prefix, suffix, events)

Listen for notifications on a bucket. Additionally one can provide filters for prefix, suffix and events. There is no prior set bucket notification needed to use this API. This is an Minio extension API where unique identifiers are registered and unregistered by the server automatically based on incoming requests.

Yields events as they occur, caller has to iterate to read these events as they occur.

Parameters

Param Type Description
bucket_name string Bucket name to listen event notifications from.
prefix string Object key prefix to filter notifications for.
suffix string Object key suffix to filter notifications for.
events list Enables notifications for specific event types.

See here for a full example.

# Put a file with default content-type.
events = minioClient.listen_bucket_notification('my-bucket', 'my-prefix/',
                                                '.my-suffix',
                                                ['s3:ObjectCreated:*',
                                                 's3:ObjectRemoved:*',
                                                 's3:ObjectAccessed:*'])
for event in events:
    print event

3. Object operations

get_object(bucket_name, object_name)

Downloads an object.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.

Return Value

Param Type Description
object urllib3.response.HTTPResponse Represents http streaming reader.

Example

# Get a full object.
try:
    data = minioClient.get_object('mybucket', 'myobject')
    with open('my-testfile', 'wb') as file_data:
        for d in data.stream(32*1024):
            file_data.write(d)
except ResponseError as err:
    print(err)

get_partial_object(bucket_name, object_name, offset=0, length=0)

Downloads the specified range bytes of an object.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.
offset int offset of the object from where the stream will start.
length int length of the object that will be read in the stream (optional, if not specified we read the rest of the file from the offset).

Return Value

Param Type Description
object urllib3.response.HTTPResponse Represents http streaming reader.

Example

# Offset the download by 2 bytes and retrieve a total of 4 bytes.
try:
    data = minioClient.get_partial_object('mybucket', 'myobject', 2, 4)
    with open('my-testfile', 'wb') as file_data:
        for d in data:
            file_data.write(d)
except ResponseError as err:
    print(err)

fget_object(bucket_name, object_name, file_path)

Downloads and saves the object as a file in the local filesystem.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.
file_path string Path on the local filesystem to which the object data will be written.

Return Value

Param Type Description
obj Object object stat info for format described below:
Param Type Description
obj.size int size of the object.
obj.etag string etag of the object.
obj.content_type string Content-Type of the object.
obj.last_modified time.time modified time stamp.
obj.metadata dict Contains any additional metadata on the object.

Example

# Get a full object and prints the original object stat information.
try:
    print(minioClient.fget_object('mybucket', 'myobject', '/tmp/myobject'))
except ResponseError as err:
    print(err)

copy_object(bucket_name, object_name, object_source, copy_conditions)

Copy a source object on object storage server to a new object.

NOTE: Maximum object size supported by this API is 5GB.

Parameters

Param Type Description
bucket_name string Name of the bucket for new object.
object_name string Name of the new object.
object_source string Name of the object to be copied.
copy_conditions CopyConditions Collection of conditions to be satisfied for the request (optional, defaults to 'None').

Example

All following conditions are allowed and can be combined together.

import time
from datetime import datetime
from minio import CopyConditions

copy_conditions = CopyConditions()
# Set modified condition, copy object modified since 2014 April.
t = (2014, 4, 0, 0, 0, 0, 0, 0, 0)
mod_since = datetime.utcfromtimestamp(time.mktime(t))
copy_conditions.set_modified_since(mod_since)

# Set unmodified condition, copy object unmodified since 2014 April.
copy_conditions.set_unmodified_since(mod_since)

# Set matching ETag condition, copy object which matches the following ETag.
copy_conditions.set_match_etag("31624deb84149d2f8ef9c385918b653a")

# Set matching ETag except condition, copy object which does not match the following ETag.
copy_conditions.set_match_etag_except("31624deb84149d2f8ef9c385918b653a")

try:
    copy_result = minioClient.copy_object("mybucket", "myobject",
                                          "/my-sourcebucketname/my-sourceobjectname",
                                          copy_conditions)
    print(copy_result)
except ResponseError as err:
    print(err)

put_object(bucket_name, object_name, data, length, content_type)

Add a new object to the object storage server.

NOTE: Maximum object size supported by this API is 5TiB.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.
data io.RawIOBase Any python object implementing io.RawIOBase.
length int Total length of object.
content_type string Content type of the object. (optional, defaults to 'application/octet-stream').
metadata dict Any additional metadata. (optional, defaults to None).

Return Value

Param Type Description
etag string Object etag computed by the server.

Example

The maximum size of a single object is limited to 5TB. put_object transparently uploads objects larger than 5MiB in multiple parts. This allows failed uploads to resume safely by only uploading the missing parts. Uploaded data is carefully verified using MD5SUM.

import os
# Put a file with default content-type, upon success prints the etag identifier computed by server.
try:
    file_stat = os.stat('my-testfile')
    file_data = open('my-testfile', 'rb')
    print(minioClient.put_object('mybucket', 'myobject', file_data, file_stat.st_size))
except ResponseError as err:
    print(err)

# Put a file with 'application/csv'.
try:
    file_stat = os.stat('my-testfile.csv')
    file_data = open('my-testfile.csv', 'rb')
    minioClient.put_object('mybucket', 'myobject.csv', file_data,
                           file_stat.st_size, content_type='application/csv')
except ResponseError as err:
    print(err)

fput_object(bucket_name, object_name, file_path, content_type)

Uploads contents from a file to objectName.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.
file_path string Path on the local filesystem to which the object data will be written.
content_type string Content type of the object. (optional, defaults to 'application/octet-stream').
metadata dict Any additional metadata. (optional, defaults to None).

Return Value

Param Type Description
etag string Object etag computed by the server.

Example

The maximum size of a single object is limited to 5TB. fput_object transparently uploads objects larger than 5MiB in multiple parts. This allows failed uploads to resume safely by only uploading the missing parts. Uploaded data is carefully verified using MD5SUM.

# Put an object 'myobject' with contents from '/tmp/otherobject', upon success prints the etag identifier computed by server.
try:
    print(minioClient.fput_object('mybucket', 'myobject', '/tmp/otherobject'))
except ResponseError as err:
    print(err)

# Put on object 'myobject.csv' with contents from
# '/tmp/otherobject.csv' as 'application/csv'.
try:
    print(minioClient.fput_object('mybucket', 'myobject.csv',
                             '/tmp/otherobject.csv',
                             content_type='application/csv'))
except ResponseError as err:
    print(err)

stat_object(bucket_name, object_name)

Gets metadata of an object.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.

Return Value

Param Type Description
obj Object object stat info for format described below:
Param Type Description
obj.size int size of the object.
obj.etag string etag of the object.
obj.content_type string Content-Type of the object.
obj.last_modified time.time modified time stamp.
obj.metadata dict Contains any additional metadata on the object.

Example

# Fetch stats on your object.
try:
    print(minioClient.stat_object('mybucket', 'myobject'))
except ResponseError as err:
    print(err)

remove_object(bucket_name, object_name)

Removes an object.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.

Example

# Remove an object.
try:
    minioClient.remove_object('mybucket', 'myobject')
except ResponseError as err:
    print(err)

remove_objects(bucket_name, objects_iter)

Removes multiple objects in a bucket.

Parameters

Param Type Description
bucket_name string Name of the bucket.
objects_iter list , tuple or iterator List-like value containing object-name strings to delete.

Return Value

Param Type Description
delete_error_iterator iterator of MultiDeleteError instances Lazy iterator of delete errors described below.

NOTE:

  1. The iterator returned above must be evaluated (for e.g. using a loop), as the function is lazy and will not evaluate by default.

  2. The iterator will contain items only if there are errors when the service performs a delete operation on it. Each item contains error information for an object that had a delete error.

Each delete error produced by the iterator has the following structure:

Param Type Description
MultiDeleteError.object_name string Object name that had a delete error.
MultiDeleteError.error_code string Error code.
MultiDeleteError.error_message string Error message.

Example

# Remove multiple objects in a single library call.
try:
    objects_to_delete = ['myobject-1', 'myobject-2', 'myobject-3']
    # force evaluation of the remove_objects() call by iterating over
    # the returned value.
    for del_err in minioClient.remove_objects('mybucket', objects_to_delete):
        print("Deletion Error: {}".format(del_err))
except ResponseError as err:
    print(err)

remove_incomplete_upload(bucket_name, object_name)

Removes a partially uploaded object.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.

Example

# Remove an partially uploaded object.
try:
    minioClient.remove_incomplete_upload('mybucket', 'myobject')
except ResponseError as err:
    print(err)

4. Presigned operations

presigned_get_object(bucket_name, object_name, expiry=timedelta(days=7))

Generates a presigned URL for HTTP GET operations. Browsers/Mobile clients may point to this URL to directly download objects even if the bucket is private. This presigned URL can have an associated expiration time in seconds after which it is no longer operational. The default expiry is set to 7 days.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.
expiry datetime.datetime Expiry in seconds. Default expiry is set to 7 days.
response_headers dictionary Additional headers to include (e.g. response-content-type or response-content-disposition)

Example

from datetime import timedelta

# presigned get object URL for object name, expires in 2 days.
try:
    print(minioClient.presigned_get_object('mybucket', 'myobject', expires=timedelta(days=2)))
# Response error is still possible since internally presigned does get bucket location.
except ResponseError as err:
    print(err)

presigned_put_object(bucket_name, object_name, expires=timedelta(days=7))

Generates a presigned URL for HTTP PUT operations. Browsers/Mobile clients may point to this URL to upload objects directly to a bucket even if it is private. This presigned URL can have an associated expiration time in seconds after which it is no longer operational. The default expiry is set to 7 days.

NOTE: you can upload to S3 only with specified object name.

Parameters

Param Type Description
bucket_name string Name of the bucket.
object_name string Name of the object.
expiry datetime.datetime Expiry in seconds. Default expiry is set to 7 days.

Example

from datetime import timedelta

# presigned Put object URL for an object name, expires in 3 days.
try:
    print(minioClient.presigned_put_object('mybucket',
                                      'myobject',
                                      expires=timedelta(days=3)))
# Response error is still possible since internally presigned does get
# bucket location.
except ResponseError as err:
    print(err)

presigned_post_policy(PostPolicy)

Allows setting policy conditions to a presigned URL for POST operations. Policies such as bucket name to receive object uploads, key name prefixes, expiry policy may be set.

Create policy:

from datetime import datetime, timedelta

from minio import PostPolicy
post_policy = PostPolicy()

# Apply upload policy restrictions:

# set bucket name location for uploads.
post_policy.set_bucket_name('mybucket')
# set key prefix for all incoming uploads.
post_policy.set_key_startswith('myobject')
# set content length for incoming uploads.
post_policy.set_content_length_range(10, 1024)

# set expiry 10 days into future.
expires_date = datetime.utcnow()+timedelta(days=10)
post_policy.set_expires(expires_date)

Get the POST form key/value object:

try:
    signed_form_data = minioClient.presigned_post_policy(post_policy)
except ResponseError as err:
    print(err)

POST your content from the command line using curl:

curl_str = 'curl -X POST {0}'.format(signed_form_data[0])
curl_cmd = [curl_str]
for field in signed_form_data[1]:
    curl_cmd.append('-F {0}={1}'.format(field, signed_form_data[1][field]))

# print curl command to upload files.
curl_cmd.append('-F file=@<FILE>')
print(' '.join(curl_cmd))

5. Explore Further
0