001package Torello.Java.Build; 002 003import Torello.Java.Additional.AppendableSafe; 004 005import Torello.Java.OSCommands; 006import Torello.Java.OSResponse; 007import Torello.Java.OSExtras; 008import Torello.Java.GSUTIL; 009import Torello.Java.StorageWriter; 010 011import java.util.Objects; 012import java.util.Vector; 013import java.util.regex.Pattern; // for the Nick-Name Checker 014import java.util.function.Predicate; // Convert Nick-Name Checker to a Prediate 015import java.io.IOException; 016 017/** 018 * This interface contains all of the interactions that occur between a Cloud-Basked 019 * Storage-Bucket, such as Google Cloud Platform or an Amazon-S3 Compatible Storage-System, and 020 * this <B STYLE='color: red;'><CODE>'.jar'</CODE> and Java-Doc Build Tools</B>. 021 * 022 * <BR /><BR />Note it is somewhat difficult to delineate, exactly, what "Synchronize my 023 * {@code '.jar'} Project-Files to the Cloud" actually means. This particular Java-Interface lists 024 * exactly what is copied over to your Storage / CDN Setup. Furthermore, this Java-Interfaces 025 * delineates exactly what HTTP & Bucket-Permission Meta-Data must be updated in order to 026 * ensure that your Java-Doc Pages and archives are publicly visible. 027 * 028 * <BR /><BR />This Jar-Project, specifically the Java-HTML {@code '.jar'} Library, was built using 029 * Google-Cloud Storage-Buckets for the vast majority of this Project's lifetime. Recently, 030 * howeveer, a move to utilize the <B>Content Delivery Network</B> (CDN) named 031 * <A HREF='https://CloudFlare.com' TARGET=_blank>CloudFlare</A> has taken place. The four stages 032 * of this Build-Tool that involve copying HTML-Files and Archive-Files (synchronizing) to a CDN or 033 * Storage-System are all listed on this particular Package's Class-File-List Web-Page. The four 034 * Build-Tool Stages are listed under "Cloud Synchronization Stage Classes". 035 * 036 * <BR /><BR />The features they export do nothing more than what are listed inside the methods 037 * provided by this Java-Interface. 038 * 039 * <BR /><BR />The contents of the class {@link CloudSync} simply contain several 040 * "utilities" needed to make the whole Synchronization-Process work. As was mentioned earlier, 041 * this entire {@code '.jar'} Project (Java-HTML) was built using GCP. For this reason, the simple 042 * to use Python-Utility offered by Google know as {@link GSUTIL} was used to do perform all of the 043 * interactions that are needed to sync files from a Terminal-Shell Window to a Cloud-Based 044 * Storage-Bucket or Content-Delivery-Network 045 * 046 * <BR /><BR />The recent addition of a CDN to host the Java-Doc Files generated by this project 047 * has meant possibily adding other forms of "Cloud-Synchronization Code." Amazon has a 048 * substantially well-know Storage-API called known as "Amazon S3", or simply "S3". At the moment 049 * build code that can run a Java-Based S3-API has not been written, although it is largely because 050 * there is still work to be done on the Java-Doc Upgrader and the CSS Tokenizer & Parser Code. 051 * 052 * <BR /><BR />Likely it will either involve streamlining a simply Java Process Class that extends 053 * {@link OSCommands} for the well-known utility 054 * <A HREF='https://rclone.org' TARGET=_blank>rclone</A>, or it will mean streamlining Amazon's 055 * giant plate of Java-Spaghetti known as the Java-API for S3 into something simple that is 056 * imported into this Java-Project's API. Eventually there will be a class named either 057 * {@code RCloneCloudSync} or one named {@code S3CloudSync} (or both!). 058 * 059 * @see GSUTIL 060 * @see OSCommands 061 */ 062public abstract class CloudSync 063{ 064 // ******************************************************************************************** 065 // ******************************************************************************************** 066 // PACKAGE-PRIVATE: BuilderRecord Instance 067 // ******************************************************************************************** 068 // ******************************************************************************************** 069 070 071 // This has not been declared final. It cannot be because due to restrictions on the order of 072 // constructor initializations, something has to be initialized after the constructor has 073 // already run to completion. This abstract-class "CloudSync" is always supposed to be 074 // instantiated by the end-user of Torello.Java.Build. The actual-selected "BuilderRecord" 075 // instance isn't created until after the user has passed the list of "CloudSync" instances to 076 // the class Config. 077 // 078 // Therefore this field cannot be instantiated by this class constructor. 079 // 080 // Therefore this field cannot be declared final. 081 // 082 // Since this field ins't final, it is not going to be public. Luckily it is only used 083 // internally by the classes inside this Torello.Java.Build package. 084 // 085 // This field is set in exactly one location - on the last line of class BuilderRecord's 086 // Constructor. It cannot be set in this class constructor because this class is constructed 087 // by the user, before any CLI processing has even occured. It's a "Chicken or the Egg" 088 // type of thing. 089 090 BuilderRecord brec = null; 091 092 093 // ******************************************************************************************** 094 // ******************************************************************************************** 095 // Public-Final Fields 096 // ******************************************************************************************** 097 // ******************************************************************************************** 098 099 100 /** 101 * One of three archives generated during the Stage-4 / Tar-Jar Stage, an archive 102 * {@code '.tar'}-File of your Root Source-Code directory may be generated, and saved to a 103 * discreet Storage-Bucket Directory in the Cloud. 104 * 105 * <BR /><BR />This behavior may be controlled by this setting. This field is initialized with 106 * the value passed to the Constructor-Parameter of the exact same name 107 * {@code 'shouldSyncMainTarGzFile'} 108 */ 109 public final boolean shouldSyncMainTarGzFile; 110 111 /** 112 * Google Cloud Storage actually allows the files saved to it's Storage-Buckets to be 113 * configured such that "all of them" are, by default, all public (or all private). 114 * There is also a way to configure a Cloud Storage-Bucket to accept granular level 115 * public / private settings for each file. 116 * 117 * <BR /><BR />If the Cloud-Storage System you are configuring is set to enforce 118 * "Bucket-Level Access-Decisions", then this field should be set {@code FALSE}. If your 119 * Storage-Service (S3, GCP or R2 to name a few) is configured to allow for individual 120 * "Object-Level Access-Control", then this field should be assigned {@code TRUE}. 121 * 122 * <BR /><BR />When this field is assigned {@code TRUE}, this Build-Tool will run a complete 123 * "Make-Public" operation on all {@code '.html'}-Files which have been recently uploaded to 124 * your Web-Domain / Storage-Bucket. 125 */ 126 public final boolean shouldRunMakePublic; 127 128 /** 129 * A "Public-Release" Cloud Storage-Bucket Directory likely has no need for "Partial-Buids" 130 * (which can also be referred to as "Developer Builds"). A Partial-Build is where only some 131 * of the Class-File Documentation {@code '.html'}-Pages are updated and synchronized to the 132 * relevant Web-Page Directory. 133 * 134 * <BR /><BR />A Partial-Build can speed up, tremendously, the process of writing code, 135 * checking it, and then viewing the output on your Java-Doc Pages. Target 136 * "Source-Distribution" Web-Domains, on the other, have no need for this setting. 137 * 138 * <BR /><BR />The current Web-Domain hosted at {@code https://JavaHTML.Torello.Directory} 139 * is not configured to accept Partial-Builds. 140 */ 141 public final boolean allowPartialBuilds; 142 143 /** 144 * This field determines whether a target Storage-Bucket should permit the Stage-8 145 * 'Set-Max-Age' (a.k.a. 'No-Browser-Cache') to run. For a Public-Release Build-Target, such 146 * as the Java-HTML Cloud-Flare Release target, the No-Browser-Cache option is not needed. 147 * As a "Developer-Release" Target, it is updated much less frequently (more infrequently?), 148 * and the Browser-Cache issue is completely irrelevant. 149 * 150 * <BR /><BR />For Storage-Bucket Directories that are updated & modified several times 151 * during the course of a single day (because the software-developer is diligently working on 152 * another fix), this <B>{@code Set-Max-Age}</B> option can be an invaluable tool to make sure 153 * that updated Java-Doc Pages are, indeed, updated - <I>not stale!!</I> 154 * 155 * <BR /><BR />The current Web-Domain hosted at {@code https://JavaHTML.Torello.Directory} 156 * is not configured to set the HTTP Meta-Data {@code CACHE-CONTROL} setting at all, because it 157 * is a "Release Target" Web-Domain. 158 */ 159 public final boolean includeSetMaxAgeBuilds; 160 161 /** 162 * The Storage Service / Web-Domain that you are configuring must be given a short, brief 163 * Nick-Name. This Nick-Name will be printed in the Command-Line Help-Menu Options-List. 164 * 165 * <BR /><BR />Note that this Nick-Name may not exceed 20 characters in length, or an 166 * {@code IllegalArgumentException} will be thrown by this class' constructor. 167 * 168 * <BR /><BR />This Nick-Name must ahere to the following Regular-Expression checker, or an 169 * {@code IllegalArgumentException} will be thrown by the constructor. 170 * 171 * <BR /><BR /><B STYLE='color: darkred;'>{@code Pattern.compile("\\w[\\w\\d-]*");}</B> 172 */ 173 public final String nickName; 174 175 /** 176 * The Cloud-Storage Target-Directory you intend to use. 177 * 178 * <BR /><BR />For Google-Cloud Storage-Buckets, this would be: 179 * {@code gs://Your-Bucket-Name/You/Target/Directory/} 180 */ 181 public final String cloudRootStorageDir; 182 183 184 // ******************************************************************************************** 185 // ******************************************************************************************** 186 // Constructor 187 // ******************************************************************************************** 188 // ******************************************************************************************** 189 190 191 private static final Pattern checkNickNameRE = Pattern.compile("\\w[\\w\\d-]*"); 192 private static final Predicate<String> checkNickName = checkNickNameRE.asPredicate(); 193 194 /** 195 * Protected-Constructor. 196 * 197 * @param shouldSyncMainTarGzFile This is a boolean that identifies whether the Project's main 198 * Backup-{@code '.tar'} File should be synchronized to a Cloud-Storage "Backup" Directory. 199 * Note that this addition can be an invaluable means for ensuring that your code is never lost 200 * or deleted (once, each and every day), without actually resorting to using Monolithic Tools 201 * like GitHub. 202 * 203 * <BR /><BR />I am not "opposed" to Git-Hub, but I simply cannot, in good conscious, use it. 204 * This HTML Project has been extended into a Code-Documentation Tool, and that means Git-Hub's 205 * features that expose your code, without including Java-Doc, make it untennable for this 206 * project. 207 * 208 * @param shouldRunMakePublic This boolean should be used to indicate whether or not the 209 * Cloud-Synchronization Stages need to ensure that your Java-Doc HTML Web-Page Files are made 210 * publicly visible. 211 * 212 * <BR /><BR />In GCP, whenever Object-Level Permissions are selected for a Storage-Bucket, it 213 * is imperative to set the "Access-Control" settings for each of the Java-Doc Web-pages are 214 * updated to allow "Read Access." However, in GCP (and other Web-Domain Storage-Services), it 215 * is also possible to assign "Bucket-Level Permissions". In such cases, there is no need to 216 * assign "Public Visible" to each of the Java-Doc {@code '.html'}-Filess that have been copied 217 * to your domain. 218 * 219 * <BR /><BR />This Constructor-Parameter is assigned to the field {@link #shouldRunMakePublic}, 220 * and that field's documentation / explanation does contain more information about this 221 * setting. Please review the information provided there. 222 * 223 * @param allowPartialBuilds Decides whether the {@code -pb} CLI Options are permitted. This 224 * configuration was formerly known as "Release or Developer", and only Developer 225 * Storage-Buckets allowed Partial-Builds to happen on their Storage-Space. 226 * 227 * <BR /><BR />When this {@code CloudSync} configuration parameter is {@code 'false'}, none of 228 * the CLI-Command-Line options for specifying a partial Package-List are included within the 229 * Main-Menu for this Cloud-Storage Target. 230 * 231 * <BR /><BR />This Constructor-Parameter is assigned to the field {@link #allowPartialBuilds}, 232 * and that field's documentation / explanation does contain more information about this 233 * setting. Please review the information provided there. 234 * 235 * @param includeSetMaxAgeBuilds The Build-Tool allows you to request that a particular 236 * Cloud-Storage Target be configured to expect that all Java-Doc {@code '.html'}-Files that 237 * are copied moved onto it have an HTTP Meta-Data {@code 'CACHE-CONTROL'} setting. 238 * 239 * <BR /><BR />When this {@code boolean}-Parameter is passed {@code TRUE}, the HTTP 240 * Configuration-Parameter {@code 'CACHE-CONTROL'} is assigned the value {@code 'public'} 241 * and {@code 'max-age=150'} to each an every Java-Doc {@code '.html'}-File that is copied 242 * to your Storage-Buckets. When you view those files in your Web-Browser (like Google Chrome 243 * for instance), the pages you view <B STYLE='color: red;'><I>will not be cached by your 244 * browser!</I></B>. 245 * 246 * <BR /><BR />This can make it worlds easier for checking, re-compiling, and re-updating 247 * your pages without having to go and erase everything you have out of your Browser-Cache. 248 * 249 * <BR /><BR />This Constructor-Parameter is assigned to the field 250 * {@link #includeSetMaxAgeBuilds}, and that field's documentation / explanation does contain 251 * more information about this setting. Please review the information provided there. 252 * 253 * @param nickName This is a required parameter that is used to facilitate your making 254 * selections at the Command-Line Interface regarding which Cloud-Build Sychronization setup 255 * you would like to use. 256 * 257 * <BR /><BR /><B STYLE='color: red;'>NOTE:</B> Early on in your project, likely, you won't 258 * need more than one Storage-Synchronization {@link CloudSync}-Instance. As your project 259 * grows, you will likely want to have a Storage-Bucket for you Java-Doc Pages that contain 260 * some recently stable-version of your {@code '.jar'}-File. 261 * 262 * <BR /><BR />Then, adding a second Web-Site, or Site-Sub-Directory for serving up pages that 263 * are still under development (not working yet) will likely reduce your stress levels and 264 * (hopefully) the impact on your blood-pressure. 265 * 266 * @param cloudRootStorageDir This is the Cloud-Based "Root Directory" for your Project. 267 * 268 * @throws NullPointerException If either {@code nickName} or {@code cloudRootStorageDir} are 269 * passed null. 270 * 271 * @throws IllegalArgumentException If the rules for a valid Cloud-Storage Target are not 272 * properly adhered, as per the explanation in the documentation for field {@link #nickName}, 273 * above. 274 */ 275 protected CloudSync( 276 final boolean shouldSyncMainTarGzFile, 277 final boolean shouldRunMakePublic, 278 final boolean allowPartialBuilds, 279 final boolean includeSetMaxAgeBuilds, 280 final String nickName, 281 final String cloudRootStorageDir 282 ) 283 { 284 Objects.requireNonNull(nickName); 285 Objects.requireNonNull(cloudRootStorageDir); 286 287 this.shouldSyncMainTarGzFile = shouldSyncMainTarGzFile; 288 this.shouldRunMakePublic = shouldRunMakePublic; 289 this.allowPartialBuilds = allowPartialBuilds; 290 this.includeSetMaxAgeBuilds = includeSetMaxAgeBuilds; 291 this.nickName = nickName; 292 this.cloudRootStorageDir = cloudRootStorageDir; 293 294 if (nickName.length() > 20) throw new IllegalArgumentException 295 ("The Target Storage-Bucket Nick-Name provided is too long: " + nickName); 296 297 if (! checkNickName.test(nickName)) throw new IllegalArgumentException( 298 "The Target Storage-Bucket Nick-Name provided [" + nickName + "] does not pass the " + 299 "Name Regular-Expression Checker: " + checkNickNameRE.toString() 300 ); 301 } 302 303 304 // ******************************************************************************************** 305 // ******************************************************************************************** 306 // public class S05_SyncJavaDoc - SYNCHRONIZE ENTIRE PROJECT 307 // ******************************************************************************************** 308 // ******************************************************************************************** 309 310 311 /** 312 * This method should delete all files in the Cloud's {@code 'javadoc/''} directory. 313 * 314 * @return The output-text and O/S Response-Code generated by invoking this method. 315 * @throws IOException throws if there are any I/O Problems thrown by this method 316 */ 317 public abstract OSResponse removeCloudJavaDocDir() throws IOException; 318 319 /** 320 * This method should copy all files from the Machine-Local {@code 'javadoc/'} directory to 321 * the {@code 'javadoc/'} directory in the cloud. 322 * 323 * @return The output-text and O/S Response-Code generated by invoking this method. 324 * @throws IOException throws if there are any I/O Problems thrown by this method 325 */ 326 public abstract OSResponse copyJavaDocDirToCloudDir() throws IOException; 327 328 /** 329 * This Method should be run if your Cloud-Storage Directory requires your recently copied / 330 * synchronized {@code'.html'} Documentation-Files to be explicity made public. 331 * 332 * @return The output-text and O/S Response-Code generated by invoking this method. 333 * @throws IOException throws if there are any I/O Problems thrown by this method 334 */ 335 public abstract OSResponse makePublicJavaDocDir() throws IOException; 336 337 338 // ******************************************************************************************** 339 // ******************************************************************************************** 340 // public class S05_SyncJavaDoc - SYNCHRONIZE SPECIFIED LIST OF PROJECT PACKAGES 341 // ******************************************************************************************** 342 // ******************************************************************************************** 343 344 345 /** 346 * Copy the primary Java-Doc Web-Page Files for a Single Java Package to the appropriate 347 * Cloud Storage Directory. 348 * 349 * @param pkgRootLocalDir The local package source-code directory to copy. 350 * @param pkgRootCloudStorageDir The Cloud-Storage Target-Directory Location 351 * @return The output-text and O/S Response-Code generated by invoking this method. 352 * @throws IOException throws if there are any I/O Problems thrown by this method 353 */ 354 public abstract OSResponse copySingleJDPackageToCloud 355 (String pkgRootLocalDir, String pkgRootCloudStorageDir) 356 throws IOException; 357 358 /** 359 * Copy additional Package Sub-Directories to the Cloud. The directories that are copied are 360 * the sub-directories for a single Java-Package Sub-Directory in the Java-Doc Directory 361 * File-System Tree. 362 * 363 * <BR /><BR /><UL CLASS=JDUL> 364 * <LI>{@code doc-files/}</LI> 365 * <LI>{@code upgrade-files/stylesheets/}</LI> 366 * <LI>{@code hilite-files/}</LI> 367 * </UL> 368 * 369 * @param ose An instance of {@link OSExtras} that contains the path to the package's 370 * source-directory location on the File-System. 371 * 372 * @param copyDirs The list of the package's sub-directories to copy. 373 * @param pkgRootCloudStorageDir The Cloud-Storage Target-Directory Location 374 * @return The output-text and O/S Response-Code generated by invoking this method. 375 * @throws IOException throws if there are any I/O Problems thrown by this method 376 */ 377 public abstract OSResponse copyOtherPackageDirsToCloudDir 378 (OSExtras ose, Vector<String> copyDirs, String pkgRootCloudStorageDir) 379 throws IOException; 380 381 /** 382 * This Method should be run if your Cloud-Storage Directory requires your recently copied / 383 * synchronized {@code'.html'} Documentation-Files to be explicity made public. 384 * 385 * @return The output-text and O/S Response-Code generated by invoking this method. 386 * @throws IOException throws if there are any I/O Problems thrown by this method 387 */ 388 public abstract OSResponse makePublicDirArr(String[] dirArr) throws IOException; 389 390 391 // ******************************************************************************************** 392 // ******************************************************************************************** 393 // public class S06_SyncTarJar 394 // ******************************************************************************************** 395 // ******************************************************************************************** 396 397 398 /** 399 * This method should copy the following files to their relevant Cloud Storage Directory: 400 * 401 * <BR /><BR /><UL CLASS=JDUL> 402 * <LI>{@code Project.jar}</LI> 403 * <LI>{@code Project-JavaDoc.tar}</LI> 404 * </UL> 405 * 406 * @return The output-text and O/S Response-Code generated by invoking this method. 407 * @throws IOException throws if there are any I/O Problems thrown by this method 408 */ 409 public abstract OSResponse copyJDTarAndJarToCloud() throws IOException; 410 411 /** 412 * This Method should be run if your Cloud-Storage Directory requires your recently copied / 413 * synchronized {@code '.tar'} & {@code '.jar'} files to be explicity made public. 414 * 415 * @return The output-text and O/S Response-Code generated by invoking this method. 416 * @throws IOException throws if there are any I/O Problems thrown by this method 417 */ 418 public abstract OSResponse makeJDTarAndJarPublic() throws IOException; 419 420 /** 421 * This method should copy your {@code Project-Backup.tar} File to the appropriate 422 * date-based Cloud-Storage Backup-Directory. 423 * 424 * @return The output-text and O/S Response-Code generated by invoking this method. 425 * @throws IOException throws if there are any I/O Problems thrown by this method 426 */ 427 public abstract OSResponse backupMainTarGzFile() throws IOException; 428 429 430 // ******************************************************************************************** 431 // ******************************************************************************************** 432 // public class S07_SyncLogs 433 // ******************************************************************************************** 434 // ******************************************************************************************** 435 436 437 /** 438 * Copies all files in the User-Supplied {@code 'logs/'} Local-Directory to the Cloud 439 * Storage Log-Directory. 440 * 441 * @return The output-text and O/S Response-Code generated by invoking this method. 442 * @throws IOException throws if there are any I/O Problems thrown by this method 443 */ 444 public abstract OSResponse copyLogDirToCloud() throws IOException; 445 446 /** 447 * If there is any need to modify the HTTP {@code CONTENT-TYPE} Meta-Data, this method should 448 * be the one for doing that. 449 * 450 * @return The output-text and O/S Response-Code generated by invoking this method. 451 * @throws IOException throws if there are any I/O Problems thrown by this method 452 */ 453 public abstract OSResponse setCloudLogsContentType() throws IOException; 454 455 /** 456 * This Method should be run if your Cloud-Storage Directory requires your recently copied / 457 * synchronized Log-Files to be explicity made public. 458 * 459 * @return The output-text and O/S Response-Code generated by invoking this method. 460 * @throws IOException throws if there are any I/O Problems thrown by this method 461 */ 462 public abstract OSResponse makeLogsPublic() throws IOException; 463 464 465 // ******************************************************************************************** 466 // ******************************************************************************************** 467 // public class S08_SetMaxAge 468 // ******************************************************************************************** 469 // ******************************************************************************************** 470 471 472 /** 473 * This method should set the Meta-Information {@code 'CACHE-CONTROL'} setting for the 474 * User's Browser-Cache to zero or something very low. When writing Java-Code, if your 475 * javadoc pages are fequently being updated, tested and evaluated, making sure that the 476 * Web-Browser is not caching stale HTML-Pages from you JavaDoc Web-Page Web-Server really 477 * needs to be a very high priority thing to worry about. 478 * 479 * @return The output-text and O/S Response-Code generated by invoking this method. 480 * @throws IOException throws if there are any I/O Problems thrown by this method 481 */ 482 public abstract OSResponse setMaxAgeAll() throws IOException; 483 484 /** 485 * This does the same thing as {@link setMaxAgeAll}, but it is only executed on the pages 486 * that are specified by the recently updated packages array. 487 * 488 * <BR /><BR />I am going to explain this simply little thing later on. 489 * 490 * @return The output-text and O/S Response-Code generated by invoking this method. 491 * @throws IOException throws if there are any I/O Problems thrown by this method 492 */ 493 public abstract OSResponse setMaxAgeSome() throws IOException; 494 495 496 // ******************************************************************************************** 497 // ******************************************************************************************** 498 // Protected Log-Printing Stuff 499 // ******************************************************************************************** 500 // ******************************************************************************************** 501 502 503 /** Protected, Internal Method. Initiates a Build-Stage's Log-Output Printing Mechanism. */ 504 protected abstract void initStage(Appendable logOnly, Appendable logAndScreen); 505 506 /** 507 * Protected, Internal Method. Initiates "Synchronie Logs" Build-Stage Output-Printing 508 * Mechanism 509 */ 510 protected abstract void initStageLogSync(); 511 512 /** Finalizes the a particular Build-Stages Log Printing Output. */ 513 protected abstract void endStage(); 514}