(SrSSKJr SSKJr SSKJr SSKJr SSKrSSKJs J r SSK J r SSK J r SS K Jr SS KJr "S S \ R$5rg) aHTests for parallel uploads ported from gsutil naming tests. Currently, the mock storage service is not thread-safe and therefore not suitable for multiprocess/multithreaded testing. Since parallel composite uploads necessarily create at least one worker thread outside of main, these tests are present in this file as temporary (slower) integration tests to provide validation for parallel composite uploads until a thread-safe mock storage service rewrite. Tests for relative paths are not included as integration_testcase does not support modifying the current working directory. )absolute_import)print_function)division)unicode_literalsN) SkipForS3) ObjectToURI)SequentialAndParallelTransfer)Retryc\rSrSrSr\S5r\S5r\S5r\S5r \S5r \ "S5\S 55r \S 5r \S 5r\S 5r\S 5rSrg)TestParallelCp1z#Unit tests for gsutil naming logic.cURSS9nUR5nURSU[U5/5 UR US5nUR [US5US5 g)z-Tests copying one top-level file to a bucket.f0 file_namecprNCreateTempFile CreateBucket RunGsUtilsuriAssertNObjectsInBucket assertEqualselfsrc_filedst_bucket_uriliness /platform/gsutil/gslib/tests/test_parallel_cp.pytestCopyingTopLevelFileToBucket.TestParallelCp.testCopyingTopLevelFileToBucket4sj""T"2H&&(NNND(D$89:  ' ' :ET.$/q:c:URSS9nURSS9nUR5nURSX[U5/5 UR US5nUR [US5US5 UR [US5US5 g) z)Tests copying multiple files to a bucket.rrf1rrrNr)r src_file0 src_file1rrs r testCopyingMultipleFilesToBucket/TestParallelCp.testCopyingMultipleFilesToBucket>s##d#3I##d#3I&&(NNND)^0DEF  ' ' :ET.$/q:T.$/q:r#cUR5n[RRUS5n[R"U5 UR USSS9nUR 5nURSU[US5/5 URSU[US5/5 URUS5nUR[US5US5 UR[US 5US 5 g ) zTests copying a nested file to a bucket subdir. Tests that we correctly translate local FS-specific delimiters ('' on Windows) to bucket delimiter (/). subdirobjr#)tmpdirrcontentsrzsubdir/ar&rz subdir/objrN) CreateTempDirospathjoinmkdirrrrrrr)rr.r,rrrs r #testCopyingNestedFileToBucketSubdir2TestParallelCp.testCopyingNestedFileToBucketSubdirJs   !F WW\\&( +FHHV""&EC"PH&&(NNND(D$DEFNND(D$BCD  ' ' :ET.*5uQx@T.,7qBr#c UR5nUR/SQS9nURSSU[U5/5 [R R U5SnURUS5nUR[XSSS 5US 5 UR[XS 5US5 UR[XS 5US 5 UR[XS5US5 g)z>Tests recursively copying absolute path directory to a bucket.)rr%f2.txt)dir0dir1nested test_filesr-Rrr9r:r;rrr%r&r8N) rr0rrr1r2splitrr)rr src_dir_root src_tmpdirrs r "testCopyingAbsolutePathDirToBucket1TestParallelCp.testCopyingAbsolutePathDirToBucket^s&&(N%%E&GLNND$ d>.BCD|,Q/J  ' ' :ET.ffhO1XT.d;U1XFT.d;U1XFT.h?qJr#c "UR5nURS/S9nURSS[RR USS5[ U5/5 URUS5nUR[ USS5US 5 g ) zTests copying a directory containing 1 file to a bucket. We test this case to ensure that correct bucket handling isn't dependent on the copy being treated as a multi-source copy. )r9r:foor<rr>r9r:rrGrN) rr0rr1r2r3rrr)rrsrc_dirrs r 'testCopyingDirContainingOneFileToBucket6TestParallelCp.testCopyingDirContainingOneFileToBucketns&&(N  -D,E FGNN d  Wff- ^  ' ' :ET.&%8%(Cr#z]The boto lib used for S3 does not handle objects starting with slashes if we use V4 signaturecURSS9nUR5nURSU[U5S-/5 UR US5nUR [U5S-US5 g)zATests copying a file to an object containing consecutive slashes.rrrz//objrrNrrs r -testCopyingFileToObjectWithConsecutiveSlashesr#c UR5nURUSSS9 URS/S9nUR5nURSS[ US5U<[ R <S3[ U5/5 URUS 5nUR[ US5US 5 UR[ US5US 5 g ) z,Tests copying objects and files to a bucket.r%foo object_namer/f2r<rr>**r&rrN) r CreateObjectr0rrr1seprr)rsrc_bucket_urirHrrs r testCopyingObjsAndFilesToBucket.TestParallelCp.testCopyingObjsAndFilesToBuckets&&(Nn$H  TF 3G&&(NNN d ^T"RVV$ ^   ' ' :ET.$/q:T.$/q:r#c UR5nURUS-SS9 UR5nURSSUS-[ US5/5 UR US5nUR [ US 5US 5 g ) a7Tests copying a directory with a single file recursively to a bucket. The file should end up in a new bucket subdirectory with the file's directory structure starting below the recursive copy point, as in Unix cp. Example: filepath: dir1/dir2/foo cp -r dir1 dir3 Results in dir3/dir2/foo being created. z /dir1/dir2rG)r.rrr>z/dir1dir3rz dir3/dir2/foorN)r0rrrrrr)rrHrrs r -testCopyingSubdirRecursiveToNonexistentSubdirTRS[TST-S5/SS9nURS5nTRS[ U55 TR[TST-S5US 5 TR[TST-S 5US 5 TR[TST-S 5US 5 TR[TST-S5US5 g)z7Validate files were copied to the correct destinations.lsrarST) return_stdout existingrrrr%r&rRr@N)rrrArlen)stdoutrrirs r _Check1HTestParallelCp.testCopyingWildcardedFilesToBucketSubDir.._Check1s>>T.*q.$?@ ""&,,t$%   1c%j )   4 Q K 8 %   4 QEuQx P   4 QEuQx P   4 QEuQx Pr#N) rrTr0rangerr1r2r3rr AssertionError)rfinal_dst_charrHrmrrls` @@r (testCopyingWildcardedFilesToBucketSubDir7TestParallelCp.testCopyingWildcardedFilesToBucketSubDirs$((*n $6!') $6!')"".@"AgQx!   GGLL$ ' a 0> A  ~QQ 7 Q 8 Q  +$r#c PSGHnUR5nURUSSS9 URUSSS9 [S5HBnURS/S9nUR S S [ US 5[ US U-5U-/5 MD UR US 5nUR[ USS SSS5US5 UR[ USS5US5 UR[ USS SSS5US5 UR[ USS5US5 GM" g)z1Tests copying one nested file to a bucket subdir.r^zd0/placeholderrOrPzd1/placeholderr&)d3d4r;r%r<rz-rruzd%dr?d0rvr;r%r placeholderrd1r@N)rrTror0rrrr)rrqrrlrHrs r &testCopyingOneNestedFileToBucketSubDir5TestParallelCp.testCopyingOneNestedFileToBucketSubDirsH$((*n $4!') $4!')Qx!$$1M0N$O  $ $   +n <  )).!rsC, '%' ''?0:(DLX77DLr#