Abstract
Cloud Computing is widespread to support industrial and academy related applications worldwide. The archival data process in the cloud is one of the challenging tasks. To handle this issue, the authors addressed the model, which allows authenticating the cloud storage using the JOSS (Java Library for Open stack storage) through the authentication URL. To achieve the authentication of the heterogeneous data while uploading into the cloud storage, a proposed model developed using the JOSS. To address this research question and demonstrate our approach, we have created the API for the archival solution in order to authenticate the cloud storage using the JOSS authentication model, providing the authentication URL. It consists of the user id and password for the verification of the user while transmitting the data into the cloud storage. The API is becoming dynamic and complex, and one of the major challenges faced by the developer in order to develop appropriate standalone applicataion interface. The extensible mechanism for authentication requires a URL in order to connect to the cloud storage system. The objective of this paper is to enhance/optimize the rate of transmission to upload the data in the cloud storage using the rclone.
Introduction
The main idea to implement our approach is to use the JOSS authentication model using the authentication URL of the cloud storage and implement it by using ‘rclone’ in the Perl script as part of the bulk file data transfer process. It allows the user to create an extensible, modular defined architecture for providing the authentication for the user to connect to the cloud data storage while on run-time bulk data uploading process. In order to address the main research question, our approach contributes to the domain of cloud, data managing capabilities of service-based cloud platforms. Our main contributions include the approach itself, which introduces the JOSS authentication model using an authentication URL of the cloud storage while connecting to the cloud storage. The basic authentication of the cloud storage URL in order to communicate with the cloud storage container as part of the data of the ‘rclone’ configuration and that can be configurable by the user with the help of ‘rclone’ configuration. The authentication URL can be provided as part of the ‘rclone’ configuration while the user tries connecting to the container of the cloud storage. It can be taken care of by ‘rclone’ utility in the underlying configuration based on the input parameters provided by the user as part of the ‘rclone’ configuration. The input parameter would be the authentication URL and can be configurable by the user, and also, the other input parameter would be the container details basically where the files will be uploaded provided in the configuration file, which the Application uses as part of the upload process. During the last decade, cloud computing is a tremendous concept basically known to Information Technology (I.T.) professionals and computer scientists to a widely-known and recognizable by ordinary computer users. While industries are anticipating more and more economic benefits from migrating and running I.T. tasks in cloud storage environments, because of so widely accepted features in the cloud technologies, most of the organizations are using cloud storage. For bulk data transfer and maintain them in the cloud, however, based on the organization’s interest and download the files from the cloud as and when required. The Application uploads the larger heterogeneous data in the cloud storage with optimized data in cloud storage. Every organization requires the data to store for the archival. In order to upload the data, the important feature is the authentication model using the JOSS model, by using the JOSS authentication model. The user account will be created in order to access the container in cloud storage, using the authentication URL the user will be authenticated by validating the user id and password provided as part of the ‘rclone’ configuration. In a daily basis, we create or receive some new data from other sources or vendors, but in our local system does not have much capacity to store the larger data, in order to transmit the large data to the cloud storage, the JOSS authentication is implemented which needs the input parameter here as the authentication URL and also the user-id followed by the password for the verification of the user. The authentication URL is provided as one of the input parameters for the basic authentication to make it happen. So our archival solution API does this capability of authenticating the storage container resides in the cloud the entire authentication process is being taken care of the ‘rclone’ configuration. The API has the capability of accessing the storage container that resides in the cloud, which is user-configurable; each user will provide the user id followed by the password. The user can configure the parameter as part of the ‘rclone’ configuration, and the container information will be passed in the configuration file; that configuration file will be taking the container information from the configuration file in which the Application reads the input parameter. When a customer chooses to use cloud services for the data transmission, before that the authentication is required for each user to access in order to maintain the security, so here the author has used the JOSS authentication model and providing the authentication URL for the user and user id followed by the password. The JOSS authentication model plays a very important role as part of the data transmission using ‘rclone’ utility. Padhy et al. [1] discussed the utility of an object-oriented metrics component, which examines the feasibility of. Net and C# object-oriented program from the perspective of mobile learning. Padhy et al. [2] discussed the reusable software components from the component-based system. They have explored several components from the standard repositories. Padhy et al. [3] examine several algorithms which able to estimate the software metrics. They developed the software metrics algorithms and pro.osed model using novel evolutionary algorithms.
This paper consists of 9 sections, Section 2. Explores the related work related to the automation API for authentication and security for file uploads in the cloud Storage environment, Section 3 focuses more on research objective and outcomes, Section 4 emphasizes the research questions, Section 5 focus more on the data collection process, Section 6 experimental observations, and proposed algorithms, configuration details, proposed architecture. Section 7 describes the research contribution, and Section 8 emphasizes more on expected output, Section 9 experimental observations and finally, Section 10 draws the conclusion and proposed future works.
For Summarized data transfer tools and its data transfer rate
For Summarized data transfer tools and its data transfer rate
In this paper, we discussed the JOSS authentication model, which helps for handshaking as part of the connectivity test to the cloud storage. As a result of which the data can be transferred from the local system to any of the cloud storage. In our literature survey, we studied the existed work of this model. One of the most challenging problems of cloud services is to authenticate the user for the security while transmitting the heterogeneous data from the local storage systems to cloud storage systems as part of the archival process. In order to authenticate the user for the data upload to the cloud storage system, the JOSS basic authentication model is being implemented as part of the ‘rclone’ configuration. In this authentication process, we follow a URL that has a user I.D. and password for validation. After verification, the data can be uploaded and accessed by the remote storage of the cloud. The other input parameter can be used for the container details provided in the configuration file, which can be read through the Application. In order to facilitate a better understanding, we provide a better authentication approach for validating cloud storage using the standalone API. In our research, we develop a new automation API that authenticates automatically by the right user and provides details for the configuration file and upload process to the cloud storage instead of the existing model. The input parameters here basically the authentication URL and the user details as part of the ‘rclone’ configuration file, and other parameters would be the container details which were provided by the Application based configuration file, which is user-configurable. Husain et al. [4] developed one interface which allows to integrate the isolated, fragmented data and service the mashed information via human and machine interfaces in a secure, scalable manner. Mazumdar et al. [5] presented a paper related to cloud- big data. They have done a systematic literature review. The objective of this extensive survey about data storage and placement in the cloud system. Strohbach et al. [6] discussed big data storage mechanism. They have pointed out the difficulties of Data storage in big data. Not only the researchers focus on the storage of data in the cloud, but they also need to concentrate on a distributed environment. As the heterogeneous information available in the cloud and utmost care needs while uploading the bulk data and their security. Minowa et al. [7] presented the paper about the techniques of upload the bulk data and proposed many schemes and algorithms to secure the data in the distributed environment. José Luis et al. [8] presents the Cost-aware Multi Data-Center Bulk Transfers in the cloud from a Customer-Side Perspective. They have used different cloud applications and presents regarding the costs of wide-area bandwidth; the overall cost of inter-data-center communication is a major concern. Rivera et al. [9] discussed Navigating the Unexpected Realities of Big Data Transfers in a Cloud-based World. Vrable et al. [10] discussed the file system along with the backup storage management system. The author proposed several algorithms and models to handle backup storage management in cloud computing. Buyya et al. [11] represented a cloud platform for the industry, along with their Aneka enterprise Cloud technology. Their objective was to realize the market-oriented resource allocation. The author’s prime focus was the comparative study between HPC (High-Performance Computing) and Internet-based services workload. Buyya et al. [12] discussed the importance of cloud computing and its implementation details. The author described the tools and techniques, implications and future scope of cloud computing. The below mentioned Table 1 describes the different uploading schemes. There has been much exertion in the exploration network coordinated at contemplating savvy mass information move over the Internet [13, 14, 15]. The greater part of these works is in the setting of well known Internet Service Provider (I.S.P.) valuing models that bill customers dependent on the 95th percentile of transfer speed use. These works at that point center around how to calendar mass information move, so the 95th percentile use is not altogether expanded. Be that as it may, the transfer speed valuing models of open C.S.P.s’ profoundly varies from I.S.P.s. Most open C.S.P.s charge customers a level expense for each byte for traffic outbound from cloud server farms. Inbound traffic and intra-data-focus traffic is normally free. Enhancement systems proposed under the 95th percentile use model in this manner don’t legitimately apply. We mention two objective facts about C.S.P. evaluating arrangements. In the first place, the expense per byte changes generally dependent on the source server farm, and whether the beneficiary is an inner EC2 server farm or an outside server farm,e.g., information might be repeated over various cloud suppliers, or the cloud and an on-premise server farm. For instance, inside the Amazon EC2 foundation, the expense per G.B. ranges from $0.02 to $0.16 dependent on the source server farm for correspondence inside EC2 – costs are increasingly costly for outer recipients. Second, C.S.P.s regularly offer limits once the outbound traffic of a client in a particular server farm surpasses some fixed limit over a month. The Internet people group has proposed a few components to cut operational consumptions of online administration organizations a considerable lot of which may likewise apply to C.S.P.s. Rather than these works, our emphasis is on diminishing expenses of clients of C.S.P.s. Earlier work [16] has called attention to that the expense of power changes with the time of day and crosswise over areas and investigated moving calculation to server farms that are less expensive at a given time. Essentially, a few investigations center around how I.S.P. suppliers charge discount customers for transmission capacity [17]. In order to upload/download the files from the cloud system to a mobile system, and vice versa through 5G networks having several issues.
As 5G having complex infrastructure, it requires a large number of base stations. Suffering the problem with data will transfer at a high rate. Suffering the problem with energy consumption.
In order to avoid these issues during file transfer, requires Cognitive Radio Networks (C.R.N.) and Massive Multiple-Input and Multiple-Output (mMIMO), and high-speed architecture required. Again to enhance the efficiency mMIMO requires the large number of antenna which will further be communicating devices
The main objective of building our approach is to authenticate the user before uploading the data from the local system into cloud storage. In order to achieve storage access authentication, the JOSS authentication model for the user was enhanced in order to provide an authenticate the data transfer into the cloud storage in run time by the automated API and how the authentication was provided by using JOSS authentication model. An API was developed to authenticate the user by taking the input parameter provided in the configuration file. So in order to achieve the automation, API was researched and developed to have that capability to provide authentication to the user. Our main contributions include the approach itself, which allows authentication to the user while accessing to the cloud storage and then transmitting data in an extensible and scalable manner using the prototype of the API, and a ‘rlcone’ sync technology. The main objectives building our approach is to optimize the data transfer from the local system into the cloud storage, In order to achieve data optimization, the configuration file usability methodology was enhanced in order to optimize the data transfer into the cloud storage within a very short span of time, how the data are being transmitted into the cloud storage is pretty much depends on the user’s configurable ‘rclone’ parameter (transfers), so in order to achieve the automation API was researched and developed to have that capability to transmit the huge data and how effectively the data will be transmitted into the cloud system using Perl script, Our main contributions include the approach itself, which allows more data optimization of data into the cloud storage and transmitting data in an extensible and scalable manner using the prototype of the API, and the rlcone sync technology
Research questions
In this paper, the author addressed the important research questions:
.
How to achieve the authentication of the heterogeneous data while uploading into the cloud storage by using the JOSS authentication model by developing an automation API (Application Programming Interface) to authenticate the user while uploading the data from the local storage system to the cloud.
.
As part of the research question #1 (How to achieve the authentication of the heterogeneous data while uploading into the cloud storage?), After extensive search from the literature review, we found that there are still some challenges identified and which can be implemented as part of the security, In this research question a user gets authenticated using the user, password, key and the endpoint authentication URL, so here we have some challenges identified as part of the data level security once a user is authenticated and tries to transmit the data to the cloud storage.
.
How to achieve the authentication of the heterogeneous data while uploading it into the cloud storage?
.
As part of the research question #2 (How to enhance/optimize the rate of transmission to upload the data in the cloud storage using the rclone?), After extensive search from the literature review, we found that there are still some challenges identified and which can be implemented as part of the data transmission rate optimization, in order to achieve we are configuring the input parameters via ‘rclone’ utility separately, instead of configuring the input parameters for the transmission rate separately, we can automate the configuration which an application can handle in order to get some more transmission rate w.r.to the data upload rate into the cloud storage
.
How can we store and maintain the heterogeneous data in the cloud storage developing an automation API?
.
As part of the research question #3 (How to achieve the authentication of the heterogeneous data while uploading into the cloud storage?), After extensive search from the literature review, we found that there are still some challenges identified and which can be implemented as part of the security, In this research question a user gets authenticated using the user, password, key and the endpoint authentication URL, so here we have some challenges identified as part of the data level security once a user is authenticated and tries to transmit the data to the cloud storage.
In order to address this research question and demonstrate our approach, we have created the automation API for the archival solution to authenticate the user before uploading the data into the cloud and discussed RQ1.
The API was built using the below technologies, and as part of the API development, the Application uses a Perl script that uses the rclone sync technology to transfer the data effectively in an organized and optimized way of data transfer.
The Application also uses JOSS (Java Library for Open Stack Storage) for authentication from the cloud) for authentication and the handshaking purposes.
Using Perl Script. Rclone sync for data transmission. JOSS (Java Library for Open Stack Storage) for authentication from the cloud).
In the field of present computing, one of the challenges is to build the automatic application upload process. An important issue for this Application is to achieve the upload automation process and provide the basic authentication in order to access the cloud storage using JOSS authentication methodology in order to provide authentication to the user to access storage in the cloud. In order to acquire the context of the user authentication, the basic authentication JOSS methodology was followed to implement and has been examined for the automation authentication process. An authentication based model JOSS was proposed which provides the authentication for the user before the upload process to the cloud system. In order to extract that data for our Application to use, we have used the basic authentication model called JOSS. Which is going to extract data from the local system and data transfer rate between the local storage system and the cloud storage? As part of the JOSS authentication model, the authentication URL is taken as one of the input parameters through which the handshake happens with the cloud storage before the upload process starts as part of the data archival. The other input parameters are the container details provided in the configuration file, which Application reads the input parameter and stores the data under the container in the cloud storage.
Authentication with cloud storage.
As part of the JOSS authentication model, A call will be triggered from JOSS with the user name and the password directly to the cloud storage, upon receive the request it validates the user details for the authentication, then the cloud storage will provide an authentication endpoint URL through which the handshaking happens between the cloud local system and the cloud storage, and the authentication token passed in each of the call being made. Whenever any unauthorized request is being made, then it throws a (JOSS exception with 404 errors). The endpoint URL is being configured as part of the authentication process is described in the below.
Rclone
Authors referred to the particular tool, which is called an agnostic tool, which allows the copy and directories to and from 25 several storage devices as well as some protocols. It not only provides this facilitates but also a large amount of storage system, which allows a better throughput of data transfers. It is newly introduced, called a graphical user interface that provides a broad range of operations.rclone allows some core data transfer simultaneously, but by default, only 4 files can simultaneously transfer. There are different types of cloud storage available like Google Drive or Dropbox, where rclone allows the facilities to choose the chunk size value. If the file is too large, then divide it into multiple chunks. The objective of rclone is to increase the uploading schemes faster rate.
Estimation of chunk size.
As part of the JOSS authentication model, the authentication URL, user id, password are taken as input parameters and how it is being configured as part of ‘rclone’ configuration is given below.
For configuration
Step #1
D:\app>rclone config
2018/07/09 18:41:34 NOTICE: Config file "C:\\Users\\35288\\.config\\rclone\\rclo
ne.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
Step #2 select n and press enter
n/s/q> n
Step #6 Type the user id
user> user --> User name
API key or password (OS_PASSWORD).
Step #7 Write the passwd as below
key> mwuMBs48Y9gB7fq
Step #8 Write the Authentication URL as below(just copy and paste this URL - https://example.com/auth/v1.0) -> Authentication endpoint URL
Authentication URL for the server (OS_AUTH_URL).
Choose a number from below, or type in your own value
1 / Rackspace US
\ "https://auth.api.rackspacecloud.com/v1.0"
2 / Rackspace UK
\ "https://lon.auth.api.rackspacecloud.com/v1.0"
3 / Rackspace v2
\ "https://identity.api.rackspacecloud.com/v2.0"
4 / Memset Memstore UK
\ "https://auth.storage.memset.com/v1.0"
5 / Memset Memstore UK v2
\ "https://auth.storage.memset.com/v2.0"
6 / OVH
\ "https://auth.cloud.ovh.net/v2.0"
One more input parameter is being passed to the Application used configuration file, the details is given below.
Input parameter passed to the Application
#Log file path for ARCHIVAL APPLICATION
#SOURCE
SOURCE = file:///C:/CMS/MH/TxDO/STMT/
#THIS IS THE NAME OF THE CLOUD USER OR ACCOUNT
ACCOUNT = 2017 -> Container details
#DESTINATION NAMESPACE (NOTE THERE IS NO LEADING SLASH)
#NOTE THE ACCOUNT AND CONTAINER ARE NAMED THE SAME FOR THE MIGRATED DATA
#DESTINATION NAMESPACE
DEST_NAMESPACE = 2017/APP_DATA/
#THREADS
THREADS = 16
#NUMBER OF MONTHS TO BE SYNCED AND RENAME THE FOLDERS AS _SYNCED. THIS IS CALCULATED FROM THE CURRENT DATE.
STOP_MONTH = 1
LAST_SYNC = C:/app/last_sync_stmt.txt
This Algorithm illustrates how to authenticate the user by taking the input parameter (endpoint URL, user name, and password) from the ‘rclone’ configuration and the other input parameter such as (container account details under which data will be stored in the cloud storage).
Algorithm
The Application reads the input parameter (container details as an account) form the configuration file.
For handling the authentication user
For handling the authentication user
(1) declare variable config_filename
(2) If config_filename = the location of the path for configuration file
(3) read config_file
(4) set config_file = read_config_file(config_filename)
(5) set log_path_app = LOG_PATH
(6) set source = SOURCE
(7) set account = ACCOUNT -> (container account details under which daa will be stored in the cloud storage)
(8) set dest_namespace = DEST_NAMESPACE
(9) set threads = THREADS set stop_month = STOP_MONTH
(10) set last_sync = LAST_SYNC
(11) set log_name = LOG_NAME set a = source
(12) initialize variables last_proc_date = 0, today = time, start_month = 0, process_till_month =
As part of the authentication process, the Application reads one of the input parameters from the configuration file deployed in the local server under the specified directory
The configuration files are deployed under the directory:
Below are the details of the configuration file for the Application to read the input parameters, and based on these input parameters, the data transfer rate can be enhanced.
Below are the details provided in the Application to read the input parameter as part of the archival process with a better transfer rate into the cloud.
System ("C:/app/rclone.exe sync $new_source $account: $new_dest_namespace -- transfers $threads --verbose --log-file $log_file_name") == 0 or die print LOG "$!’’
In this above system command which the Application uses to pass the parameters used to get the container account details using the parallel data transfer for the data to by sync into the cloud storage.
The parameters can be configurable based on the user requirement, and these parameters that the Application reads from the configuration file as part of the authentication is given below.
ACCOUNT (THIS IS THE NAME OF THE USER OR ACCOUNT Created under the
Cloud environment)
For FTP configuration details
For FTP configuration details
In the below diagram wherein the application uploads the files to the cloud storage. On a daily basis, the huge number of files are coming from various sources such as vendors, another interfacing system, and then the files are being stored in the local system, and the automation API was developed to upload the files from the local system to the storage cloud system.
Proposed uploading and downloading scheme
In order to make the reliability of a particular file, the owner of the data prepares the method, which is called HMAC for each and every encrypted file and stores the HMAC together with the encrypted file in cloud storage. Similarly, when we required to download the file, the responsibility of the owner of the data will check the HMAC is valid before the file decrypted. The above design is completely based on an encrypted format, which is initially proposed by blinded R.S.A. [9]. Here, the owner of the data requests the key manager to decrypt the blinded version of encrypted data. If they agree, then the role of the key manager is decrypt and sends the original form of the key data. For each data communication in the cloud, the author generates the two large R.S.A. prime numbers
The product can be optimized by using
Here the corresponding control key pair needs to choose.
In this proposed Algorithm indicates that the user needs to upload the required data onto the cloud storage. The file may be the image, text, etc. In order to upload the module, the user needs to follow the rules .:-Required users email and their corresponding password. If both are matched then the cloud server allows us to connect and upload the data. For security reason, the authors have used the data encryption techniques.
[h] Uploading scheme[1] users email and password
File successfully uploaded onto the cloud storage
File upload.
File download.
For maintaining the cloud Storage.
Downloading schemeusers email and password
File successfully downloaded onto the cloud storage
The above Algorithm is used for data downloading from the cloud servers. Before downloading, it will check for authenticity. Once authenticity received, then it allows establishing the connection to the cloud server and accessing the preferred data. In order to access the data from the cloud must provide the basic details like name and email id. Here the required procedure need to invoke.
The above-proposed diagram indicates the cryptographic operations. Authors discussed primarily two kinds of files one is filed metadata. Another one is policy metadata. The above architecture implements on the basing of C++ on the windows free environment. The objective of the first one is to maintain the information having file size as well as HMAC. For integrity checking, authors used encrypted file, and it is having either 8 bytes or 20 bytes of HMAC. The second type of file contains the specification of the encrypted cryptographic keys as well as a Boolean combination of keys. For identifying the uniqueness, the authors used a 4-bytes unique identifier and for representing the Boolean combinations of keys normally used the disjunctive canonical form (OR, AND). In the above-mentioned Figure, Data owner uses a different function calls like - Upload (file, policy), Download (file). Delete (policy), Renew (file, new_policy). Key managers are used for providing basic functions: - Policy Creation, Retrieving, Revoking, etc.
Proposed architecture for uploading and downloading.
Cloud storage services.
The above Fig. 7. Indicates the approach for cloud storage services. This proposed approach is a twofold approach wherein Phase-1 testing and Phase-II is called a design and decision stage. It will measure the workload of many clients. It is one type of black-box testing where the clients send the request and considered as the input string of parameters meanwhile the system observing and maintaining the external behavior. During the testing phase, it will create a specific file for particular for cloud storage and maintains the network traffic for tracing purposes. The overall proposed architecture indicates the following sequence of operations
Request (Input) enter into the testing purpose The file needs to be upload and test through the FTP server After App-under-test it will send for synchronize purpose and generate a statistics report Finally, send to the cloud storage
The API was built using technologies, and as part of the API development, the Application uses a Perl script which uses the JOSS authentication model to authenticate the user by taking the endpoint URL, user account and password details. The API was developed to take the input parameter (account) from the application configuration file, and other input parameters are used and configure as part of the ’rclone ’configuration.
Expected outcomes
The expected behavior of the API is to authenticate the user before uploading the data into the cloud storage as per the user’s needs.
For authentication before syncing the data from the local source
For authentication before syncing the data from the local source
(User authentication before syncing the data from the local source into the cloud storage) JOB STARTED AT 2019-04-09 16:58:09CFG NAME D: _sync_2019_data_files_thread0.cfgrclone.exe sync N:019010190105 2019:2019/DATA/201901/20190105/ –transfers 16 –verbose –log-file D: _LOG_20190105_20190409165809.log
In the above execution details for the Application, we are passing the input parameter (account).
From the log file, we are testing the Connectivity to the cloud storage from the local storage, we are checking here from, how the connection is getting established before the data upload happens from the local storage to the cloud storage. The account (container storage details) which has been configured and given the details.
Output shows the connectivity to cloud storage
D:\sconnectivity>ping itoc-sd-sscpci.itocservices.com
Pinging itoc-sd-sscpci.itocservices.com [10.143.6.132] with 32 bytes of data:
Reply fron 10.143.6.132: bytes=32 time=296ms TTL=251
Reply fron 10.143.6.132: bytes=32 time=277ms TTL=251
Reply fron 10.143.6.132: bytes=32 time=288ms TTL=251
Reply fron 10.143.6.132: bytes=32 time=311ms TTL=251
Ping statistics for 10.143.6.132:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip tines in m1111-seconds:
Mininun = 277ms, Maximm = 311ms. Average = 293ms
The container 2017, which was configured as part of ‘rclone’ configuration and the container 9 account, is passed as the input parameter to the configuration file. We are testing here whether the account details have been created under the cloud storage or not using a mentioned below command, as part of the output, it will display the container details if the container is successfully created or not.
Output shows the connectivity to cloud storage
D:\connectivity>rclone lsd 2017:
385 2019-04-22 08:37:06 1 2016_1
3803415733 2019-04-22 08:32:06 201812 2012
The expected behavior of the API is to show the rate of transmission of the data into the cloud storage as per the user needs to be used in the configuration file.
Case 2. (Data transfer rate optimization while copying the data from the local source into the cloud storage)
In the above execution details for the Application, we are passing the input parameter (transfer 16), meaning the Application is running with 16 different parallel threads to enhance the transmission rate.
From the log file, we have got the below observation for the data transfer rate into the cloud storage as below.
Output
In this logfile from the parallel threads, the output for the expected data transmission rate is enhanced to ’897.205 Kbytes/s’ within 7 hours of time.
Expected data transmission rate
Expected data transmission rate
2019/04/09 00:26:56 INFO : DATA_20190102083820_442389293.PDF: Copied (new)
2019/04/09 00:26:56 INFO : DATA_20190102083820_442390407.PDF: Copied (new)
2019/04/09 00:26:56 INFO : DATA_20190102083820_442397824.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442401097.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442397410.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442405792.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442400560.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442398202.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442395224.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442382966.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442403168.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442392940.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442394888.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102083820_442405594.PDF: Copied (new)
2019/04/09 00:26:57 INFO : DATA_20190102120849_29374249.PDF: Copied (new)
2019/04/09 00:26:58 INFO : DATA_20190102083820_442397478.PDF: Copied (new)
2019/04/09 00:26:58 INFO : DATA_20190102083820_442403960.PDF: Copied (new)
2019/04/09 00:26:58 INFO : DATA_20190102083820_442402491.PDF: Copied (new)
2019/04/09 00:26:58 INFO :
Transferred: 14.058G / 14.058 GBytes, 100%, 897.205 kBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 289061 / 289061, 100%
Elapsed time: 7h14m6.5s
For uploading scheme
The uploading and downloading time can be measured and called as data transmission time. It can be calculated as the data owner and storage cloud. Once File size and total time measured, further divided into two different components like the total time required to transfer the file and the data about the file (Metadata) and rest one is called the policy element. It can measure the total time transmission time for the data about the policy mechanism.
Data transmission for different file size for an uploading Scheme.
For downloading scheme
The last component is called the key management, which is the data owner time that needs to coordinate with the key manager and manages the cryptographic keys. It is necessary when the data needs to be uploading along with the control key of the particular required policy.
Data transmission for different file size for downloading the Scheme.
In the above, Fig. 9 indicates the data transmission rate require at downloading time. It has been observed that for downloading operation, the key manager who needs to obtain the data key.
The below table indicates the performance issues related to the multiple policies. When more than one policy is connected to the file, then how the file system will maintain during the uploading and downloading time. The author’s objective is very clear about fixing the file size at Megabyte during the uploading time; as a result of the two conjunctive combinations of the policies considered, and the other one is called disjunctive. Gomathy, V.,discussed Malicious node detection using a heterogeneous cluster-based secure routing protocol (HCBS) in wireless adhoc sensor networks
The conjunctive and disjunctive policy can be defined with the followings:
Required to download the required encrypted keys from the cloud storage. For decryption, required to send the key. Data key required to recover. If possible, collect the re-encrypt the data along with the control keys. In the end, required to send the newly generated encrypted keys to the cloud storage.
Apart from this, we have a policy base scheme when associated with multiple policies. We have fixed the file upload operation size is 1 M.B. We also carefully check the two different types of policies i.e conjunctive case and one on the disjunctive case.
For conjunctive issues
The below-mentioned table primarily used for the disjunctive purpose.
For disjunctive issues
Our policy-based on-time performance. The objective was to identify the time overhead of the policy. The policy is divided into three different components. These are -Data uploading/Downloading, Data transmission between the owner and the cloud storage.
The below mention Tables are describing the size of the policy and their data about the data (metadata).
For conjunctive policy
For disjunctive policy
For conjunctive policy in the metadata.
For Disjunjuctive policy in the metadata.
The above Figs 10 and 11 meant for the number of metadata transmitted and their corresponding policy number for conjunctive and disjunctives in the metadata respectably.
The study overall describes the data management, data transfer rate optimization, and the security challenges in the cloud storage environment by developing an automation API. However, there are still some challenges that can be applied as part of the archival data process. The conclusion has been subcategorized into three different sections based on each research question as follows. In recent days the archival process in the cloud one of the challenging tasks with respect to the archival automation application. When a user gets authenticated using the user id, password and key, and authentication, some of the challenges have been identified as part of the data level security. If the proposed future work described in the above would be implemented, then it will be a very critical and one of the challenging an application as part of the archival data process into the cloud that is what the current industry expects and future work. As part of the research question (How to enhance/optimize the rate of transmission to upload the data in the cloud storage using the rclone?), After extensive search from the literature review, we found that there are still some challenges identified and which can be implemented as part of the data transmission rate optimization, in order to achieve we are configuring the input parameters via ‘rclone’ utility separately, instead of configuring the input parameters for the transmission rate separately, we can automate the configuration which an application can handle in order to get some more transmission rate w.r.to the data upload rate into the cloud storage. All the research questions addressed in the Section 4.