========================================================
LITESOPH Job Submission, Monitoring and File Management
========================================================

LITESOPH Job Submission, Monitoring and File Management is a set of modules capable of handling tasks like submission of job to remote machine, tracking the status running job and transferring of files from remote to local machine.   

Features
========

1. Job Submission module supports queue (qsub, sbatch etc) as well as non-queue (bash) based job submission on remote machine through password and passwordless approach. Passwordless approach is generally used to avoid typing password repeatively for every job submission. It can also help to bypass captcha based login authentication. See :doc:`/./FAQs/pwless-ssh`

2. Monitoring and File Management modules handles the job of tracking the status as well as terminating the  running job on remote machine. File Management module handles the downloading of all files as well as selective transfer the files from remote to local machine via different transfer methods (direct transfer along with compression and split transfer).

Functionalities
================

1. Function to submit a job at local as well as remote machine

2. Function to check the running status of job submitted at local machine as well as remote machine  

3. Function to get the information about files generated by running project at local as well as remote machine.

4. Function to view different files generated by running job at local as well as remote machine.

5. Function to kill running job at remote machine.

6. Function to selectively transfer files with varying methods (direct, compress, split).


Modules, Functions and Working
===============================
LITESOPH Job Submission, Runtime Query and File Management modules depends on two main Functionalities. Function to make connection with remote machine and execute commands on it via password as well as passwordless login authentication. 

Furthermore, Runtime Query and File Management uses mainly four modules to work: 

* Module **litesoph/common/submit.py** handles methods requires to establish connection between local and remote machine, submitting jobs at local and remote machine, and downloading files. 

* Module **litesoph/common/lfm_database.py** defines are all predefined rules, methods and database required for file management and query related Functionalities.  

* Module **litesoph/gui/task_controller.py** handles methods related to runtime query for local as well as remote job submission.

* Module **litesoph/gui/views.py** handles the GUI for all backend Functionalities related to LITESOPH Job Submission, Runtime Query and File Management. 

#. **Module: litesoph/common/job_submit.py**

    - *function* **execute_cmd_local()**:  execute command locally by using argument command and directory.
    - *function* **execute_cmd_remote()**: execute command on remote machine through local machine shell.
    - *class* **SubmitLocal**: handles local job submission.
    - *class* **SubmitNetwork**: handles remote job submission.
    - *function*  **download_files_from_remote()**
        
        * get the list of files from remote directory
        * convert the list of files into file-metadata-dictionary 
        * filter the dictionary into sub-dictionary of priority-levels
        * give each file priority level according to the assigned tags of files 
        * classify the files on the basis of priority and transfer priority1 files first
        * determine the method to transfer the files
        * transfer the file

#. **Module: litesoph/common/lfm_database.py**

    - *function* **lfm_file_info_dict()**: generate a dictionary database of containing metadata information of all the possible files.
    - *varibles*: include python list and dictionary type variables different file tags used for generating master file metadata information.
    
#. **Module: litesoph/gui/task_controller.py**

    - *class* **TaskController**: handles individual task in a workflow or a task in task mode.

        - *function* **_on_run_network_button()** : handles set of functions responsible for job submission and monitoring at remote machine

            - *function*  **_on_create_remote_job_script()**
            - *function*  **_on_save_job_script()**
            - *function*  **_run_network()**
            - *function*  **_on_check_job_status_remote()**
            - *function*  **_on_check_file_status_remote()**
            - *function*  **_on_kill_job_remote()**
            - *function*  **_on_kill_job_remote()**
            - *function*  **_on_download_all_files()**
            - *function*  **_on_download_specific_file()**
        
        - *function* **_on_run_local_button()** : handles set of functions responsible for job submission and monitoring at local machine

            - *function*  **_on_create_local_job_script()**
            - *function*  **_on_save_job_script()**
            - *function*  **_run_local()**
            - *function*  **_on_check_job_status_local()**
            - *function*  **_on_check_file_status_local()**
            - *function*  **_on_view_specific_file_local()**

#. **Module: litesoph/gui/views.py**

    - *class* **JobSubPage**: handles job submission GUI page.

        - *function*  **runtime_query_local()**: handles GUI buttons related to runtime query for local job submission
        - *function*  **runtime_query_remote()**: handles GUI buttons related to runtime query for remote job submission
        - *function*  **show_run_local()**: handles GUI buttons related for remote job submission
        - *function*  **show_run_network()**: handles GUI buttons related for remote job submission