| Current File : //proc/self/root/usr/lib/python3/dist-packages/s3transfer/__pycache__/tasks.cpython-312.pyc |
�
�<�e?? � � � d dl Z d dlZd dlmZ ej e� Z G d� d� Z G d� de� Z G d� de� Z G d � d
e� Z
y) � N)�
get_callbacksc �l � e Zd ZdZ dd�Zd� Zed� � Zd� Zd� Z d� Z
d � Z d
� Z d � Z
d � Zd
� Zd� Zy)�Taskz�A task associated to a TransferFuture request
This is a base class for other classes to subclass from. All subclassed
classes must implement the main() method.
Nc � � || _ || _ | j �i | _ || _ |�i | _ || _ | j �g | _ || _ y)a�
:type transfer_coordinator: s3transfer.futures.TransferCoordinator
:param transfer_coordinator: The context associated to the
TransferFuture for which this Task is associated with.
:type main_kwargs: dict
:param main_kwargs: The keyword args that can be immediately supplied
to the _main() method of the task
:type pending_main_kwargs: dict
:param pending_main_kwargs: The keyword args that are depended upon
by the result from a dependent future(s). The result returned by
the future(s) will be used as the value for the keyword argument
when _main() is called. The values for each key can be:
* a single future - Once completed, its value will be the
result of that single future
* a list of futures - Once all of the futures complete, the
value used will be a list of each completed future result
value in order of when they were originally supplied.
:type done_callbacks: list of callbacks
:param done_callbacks: A list of callbacks to call once the task is
done completing. Each callback will be called with no arguments
and will be called no matter if the task succeeds or an exception
is raised.
:type is_final: boolean
:param is_final: True, to indicate that this task is the final task
for the TransferFuture request. By setting this value to True, it
will set the result of the entire TransferFuture to the result
returned by this task's main() method.
N)�_transfer_coordinator� _main_kwargs�_pending_main_kwargs�_done_callbacks� _is_final)�self�transfer_coordinator� main_kwargs�pending_main_kwargs�done_callbacks�is_finals �2/usr/lib/python3/dist-packages/s3transfer/tasks.py�__init__z
Task.__init__ sb � �P &:��"� '��� � � � $� "�D� �$7��!� � &�(*�D� %� -�� � � � � '�#%�D� �!��� c � � g d�}| j | j |� }dj | j j | j
j |� S )N)�bucket�key� part_number�final_filename�transfer_future�offset�
extra_argsz{}(transfer_id={}, {}))�"_get_kwargs_with_params_to_includer �format� __class__�__name__r � transfer_id)r �params_to_display�main_kwargs_to_displays r �__repr__z
Task.__repr__T s] � �
�� "&�!H�!H� � � � 0�"
� � (�.�.� �N�N� #� #� � &� &� 2� 2� "�
�
r c �. � | j j S )z8The id for the transfer request that the task belongs to)r r! )r s r r! zTask.transfer_idi s � � �)�)�5�5�5r c �2 � i }|D ] }||v s�|| ||< � |S �N� )r �kwargs�include�filtered_kwargs�params r r z'Task._get_kwargs_with_params_to_includen s3 � � �� � 7�E����)/�� ���&� 7� � r c �N � i }|j � D ] \ }}||v r� |||< � |S r' )�items)r r) �excluder+ r, �values r �"_get_kwargs_with_params_to_excludez'Task._get_kwargs_with_params_to_excludeu s= � � �� "�L�L�N� +�L�E�5��� ��%*�O�E� "� +� � r c �p � | j � | j � }| j j � sP| j |� | j
D ] } |� � | j r| j j � S S | j
D ] } |� � | j r| j j � yy# t $ r}| j |� Y d}~�_d}~ww xY w# | j
D ] } |� � | j r| j j � w w xY w)z9The callable to use when submitting a Task to an executorN)
�_wait_on_dependent_futures�_get_all_main_kwargsr �done�
_execute_mainr
r �
announce_done� Exception�_log_and_set_exception)r r) �
done_callback�es r �__call__z
Task.__call__} s( � � ;� � +� +� -�
�.�.�0�F� �-�-�2�2�4��)�)�&�1�
"&�!5�!5�
�
� ��
� �~�~� �*�*�8�8�:� � 5� "&�!5�!5�
�
� ��
� �~�~� �*�*�8�8�:� �� � +� � '� '�� *� *�� +�� "&�!5�!5�
�
� ��
� �~�~� �*�*�8�8�:� �s1 �A
C
� C4 �
C1�C,�'C4 �,C1�1C4 �4AD5c �� � dg}| j ||� }t j d| � d|� �� | j di |��}| j r| j
j
|� |S )N�datazExecuting task z
with kwargs r( )r1 �logger�debug�_mainr r �
set_result)r r) �params_to_exclude�kwargs_to_display� return_values r r6 zTask._execute_main� sv � � $�H�� � C� C� �%�
�� � � ��t�f�M�:K�9L�M�N�!�t�z�z�+�F�+� � �>�>� � &� &� 1� 1�,� ?��r c �h � t j dd�� | j j |� y )NzException raised.T)�exc_info)r? r@ r �
set_exception)r � exceptions r r9 zTask._log_and_set_exception� s) � � � � �(�4� �8� �"�"�0�0��;r c � � t d� �)z�The method that will be ran in the executor
This method must be implemented by subclasses from Task. main() can
be implemented with any arguments decided upon by the subclass.
z_main() must be implemented��NotImplementedError)r r) s r rA z
Task._main� s � � "�"?�@�@r c �� � g }| j j � D ]8 \ }}t |t � r|j |� �(|j |� �: | j
|� y r' )r r. �
isinstance�list�extend�append�_wait_until_all_complete)r �futures_to_wait_on�_�futures r r3 z Task._wait_on_dependent_futures� sc � � �� �2�2�8�8�:� 2�I�A�v��&�$�'�"�)�)�&�1� #�)�)�&�1�
2�
�%�%�&8�9r c �� � t j d| |� |D ]* } t j d| |� |j � �, t j d| � y # t $ r Y �Ow xY w)Nz-%s about to wait for the following futures %sz%s about to wait for %sz%%s done waiting for dependent futures)r? r@ �resultr8 )r �futuresrU s r rR z Task._wait_until_all_complete� sn � � � � � ;�T�7�
� � �F�
�� � � 6��f�E��
�
�� � � � �<�d�C�� �
� �
�s �'A � A*�)A*c �$ � t j | j � }| j j � D ]S \ }}t |t
� r)g }|D ]! }|j
|j � � �# n|j � }|||< �U |S r' )�copyr r r. rN rO rQ rW )r r) r �
pending_valuerW rU s r r4 zTask._get_all_main_kwargs� s� � ����4�,�,�-�� #'�";�";�"A�"A�"C� !� �C� � �-��.��� +� 3�F��M�M�&�-�-�/�2�3� '�-�-�/�� !�F�3�K� !� �
r )NNNF)r �
__module__� __qualname__�__doc__r r$ �propertyr! r r1 r< r6 r9 rA r3 rR r4 r( r r r r sc � �� � ���
6"�p
�* �6� �6� � �;�6 �"<� A� :�D�,r r c �" � e Zd ZdZd� Zd� Zd� Zy)�SubmissionTaskz�A base class for any submission task
Submission tasks are the top-level task used to submit a series of tasks
to execute a particular transfer.
c �l � | j j � t |d� }|D ] } |� � | j j � | j dd|i|�� y# t
$ rE}| j
|� | j � | j j � Y d}~yd}~ww xY w)a>
:type transfer_future: s3transfer.futures.TransferFuture
:param transfer_future: The transfer future associated with the
transfer request that tasks are being submitted for
:param kwargs: Any additional kwargs that you may want to pass
to the _submit() method
�queuedr Nr( ) r �set_status_to_queuedr �set_status_to_running�_submit�
BaseExceptionr9 �+_wait_for_all_submitted_futures_to_completer7 )r r r) �on_queued_callbacks�on_queued_callbackr; s r rA zSubmissionTask._main� s� � �$ 7� � &� &� ;� ;� =� #0���"J� �&9�
%�"�"�$�
%�
� &� &� <� <� >�
�D�L�L� C� � C�F� C�� � 7�
� '� '�� *�
� <� <� >�
� &� &� 4� 4� 6� 6��- 7�s �A"A% �% B3�.;B.�.B3c � � t d� �)aX The submission method to be implemented
:type transfer_future: s3transfer.futures.TransferFuture
:param transfer_future: The transfer future associated with the
transfer request that tasks are being submitted for
:param kwargs: Any additional keyword arguments you want to be passed
in
z _submit() must be implementedrK )r r r) s r rf zSubmissionTask._submit& s � � "�"A�B�Br c � � | j j }|r3| j |� | j j }||k( ry |}|r�2y y r' )r �associated_futuresrR )r �submitted_futures� possibly_more_submitted_futuress r rh z:SubmissionTask._wait_for_all_submitted_futures_to_complete2 sW � � !� 6� 6� I� I�� � � )� )�*;� <� �*�*�=�=�
,� !�$C�C�� ?� �! r N)r r\ r] r^ rA rf rh r( r r ra ra � s � �� -7�^
C�@r ra c � � e Zd ZdZd� Zy)�CreateMultipartUploadTaskz#Task to initiate a multipart uploadc � � |j d||d�|��}|d }| j j |j |||�� |S )aq
:param client: The client to use when calling CreateMultipartUpload
:param bucket: The name of the bucket to upload to
:param key: The name of the key to upload to
:param extra_args: A dictionary of any extra arguments that may be
used in the initialization.
:returns: The upload id of the multipart upload
)�Bucket�Key�UploadId)rs rt ru r( )�create_multipart_uploadr �add_failure_cleanup�abort_multipart_upload)r �clientr r r �response� upload_ids r rA z CreateMultipartUploadTask._mainQ si � � 2�6�1�1�
��s�
�&0�
�� �Z�(� �
�"�"�6�6� � )� )��� � 7�
� �r N�r r\ r] r^ rA r( r r rq rq N s
� �-�r rq c � � e Zd ZdZd� Zy)�CompleteMultipartUploadTaskz#Task to complete a multipart uploadc �6 � |j d|||d|id�|�� y)az
:param client: The client to use when calling CompleteMultipartUpload
:param bucket: The name of the bucket to upload to
:param key: The name of the key to upload to
:param upload_id: The id of the upload
:param parts: A list of parts to use to complete the multipart upload::
[{'Etag': etag_value, 'PartNumber': part_number}, ...]
Each element in the list consists of a return value from
``UploadPartTask.main()``.
:param extra_args: A dictionary of any extra arguments that may be
used in completing the multipart transfer.
�Parts)rs rt ru �MultipartUploadNr( )�complete_multipart_upload)r ry r r r{ �partsr s r rA z!CompleteMultipartUploadTask._mainn s4 � � )��(�(�
��� � $�e� ,�
�
�
r Nr| r( r r r~ r~ k s
� �-�
r r~ ) rZ �logging�s3transfer.utilsr � getLoggerr r? r ra rq r~ r( r r �<module>r� sX �� � � *� �� � �8� $��X� X�v[@�T� [@�|�� �:
�$�
r