Timeout in .sh file
-
@lakshmana said in Timeout in .sh file:
@black3dynamite said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@black3dynamite said in Timeout in .sh file:
pkill chromium
https://www.garron.me/en/bits/kill-all-instances-chromium-linux-command-line.html
While using the chrome is not even starting but the working chrome is being killed
We can use but need to give credentials.I need to run without admin credentials
You can do that with cURL for sure, and I expect wget.
During using Curl the file is not downloading where the link for that file is created inside the downloaded file
Are you sending the credentials with cURL?
How to do this?
https://www.cyberciti.biz/faq/curl-download-file-example-under-linux-unix/
curl http://username:[email protected]/file/path/data.tar.gz
or
curl -u Username:Password http://server1.cyberciti.biz/file/path/data.tar.gzThis .sh file will be running in my colleague System also at that time credentials cannot be shown right! So only going for Chrome
There are still better ways. Read the password from a file that can only be read by the user which is invoking the download process. You could pass credentials from stdin or simply use curl's --netrc-file parameter:
curl [your options here] --netrc-file /your/secret/and/not/world/readable/password-file http://some.link/to/the/fileThe password file looks like this:
machine <example.com> login <username> password <password>See here for more information: https://stackoverflow.com/a/27894407
I could even think about a encrypted password stored in a file that will be decrypted on the fly using stdin passing - but that's way more advanced stuff.
-
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
I would always use curl / wget / any other CLI tool before I would use Chromium (Chromium != Chrome) for a simple file download. It's a monster in terms of resource consumption.
I tried wget where credentials is asking and in curl the file is not downloading where the test file created inside that file the link is there to download the file.So only using Chrome here
Just because you tried and didn't figure it out doesn't mean it isn't the right solution. You don't have anything working right now, or you'd not need the question. We are trying to get this working in a practical, logical way, and cURL is almost certainly the right way.
-
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
I would always use curl / wget / any other CLI tool before I would use Chromium (Chromium != Chrome) for a simple file download. It's a monster in terms of resource consumption.
I tried wget where credentials is asking and in curl the file is not downloading where the test file created inside that file the link is there to download the file.So only using Chrome here
Just because you tried and didn't figure it out doesn't mean it isn't the right solution. You don't have anything working right now, or you'd not need the question. We are trying to get this working in a practical, logical way, and cURL is almost certainly the right way.
Yes I understood but trying my efforts
-
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
I would always use curl / wget / any other CLI tool before I would use Chromium (Chromium != Chrome) for a simple file download. It's a monster in terms of resource consumption.
I tried wget where credentials is asking and in curl the file is not downloading where the test file created inside that file the link is there to download the file.So only using Chrome here
Show us the command you ran. I have a feeling you’re either not following a redirect or your output option is not set correctly.
-
@stacksofplates said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
I would always use curl / wget / any other CLI tool before I would use Chromium (Chromium != Chrome) for a simple file download. It's a monster in terms of resource consumption.
I tried wget where credentials is asking and in curl the file is not downloading where the test file created inside that file the link is there to download the file.So only using Chrome here
Show us the command you ran. I have a feeling you’re either not following a redirect or your output option is not set correctly.
curl -o "Link"
-
@lakshmana said in Timeout in .sh file:
@stacksofplates said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
I would always use curl / wget / any other CLI tool before I would use Chromium (Chromium != Chrome) for a simple file download. It's a monster in terms of resource consumption.
I tried wget where credentials is asking and in curl the file is not downloading where the test file created inside that file the link is there to download the file.So only using Chrome here
Show us the command you ran. I have a feeling you’re either not following a redirect or your output option is not set correctly.
curl -o "Link"
Could you please try what I posted above? And could you please bring some light in your requirements? What kind of authentication is required by the server? Basic HTTP Auth?
-
@thwr said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@stacksofplates said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
I would always use curl / wget / any other CLI tool before I would use Chromium (Chromium != Chrome) for a simple file download. It's a monster in terms of resource consumption.
I tried wget where credentials is asking and in curl the file is not downloading where the test file created inside that file the link is there to download the file.So only using Chrome here
Show us the command you ran. I have a feeling you’re either not following a redirect or your output option is not set correctly.
curl -o "Link"
Could you please try what I posted above? And could you please bring some light in your requirements? What kind of authentication is required by the server? Basic HTTP Auth?
The process here is the internal site is there where the .csv files will be downloaded from the site through wget or curl or by browser.Where i tried from the wget i got the error as
"HTTP request sent, awaiting response... 401 Unauthorized"-->I think this for the AD username and password where the credentials part is not even asking.
How to try the above thing which was mentioned already
-
@lakshmana said in Timeout in .sh file:
@stacksofplates said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
I would always use curl / wget / any other CLI tool before I would use Chromium (Chromium != Chrome) for a simple file download. It's a monster in terms of resource consumption.
I tried wget where credentials is asking and in curl the file is not downloading where the test file created inside that file the link is there to download the file.So only using Chrome here
Show us the command you ran. I have a feeling you’re either not following a redirect or your output option is not set correctly.
curl -o "Link"
RIght, we've known all along that that was not going to work. Stick to one thing, only look at cURL. Don't try to use Chrome, don't spend time trying to convince us that Chrome will work and cURL will not. You didn't include the credentials with cURL so of course it didn't work. If you look at the directions above, how to include the credentials was explained.
-
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@stacksofplates said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
I would always use curl / wget / any other CLI tool before I would use Chromium (Chromium != Chrome) for a simple file download. It's a monster in terms of resource consumption.
I tried wget where credentials is asking and in curl the file is not downloading where the test file created inside that file the link is there to download the file.So only using Chrome here
Show us the command you ran. I have a feeling you’re either not following a redirect or your output option is not set correctly.
curl -o "Link"
Could you please try what I posted above? And could you please bring some light in your requirements? What kind of authentication is required by the server? Basic HTTP Auth?
The process here is the internal site is there where the .csv files will be downloaded from the site through wget or curl or by browser.Where i tried from the wget i got the error as
"HTTP request sent, awaiting response... 401 Unauthorized"-->I think this for the AD username and password where the credentials part is not even asking.
How to try the above thing which was mentioned already
So how does the site work, it just has a normal login form?
-
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
-
@scottalanmiller @stacksofplates @thwr
Is there a way to prompt the user for the credentials instead?
-
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
-
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
That .sh file should download the data by terminal or by Browser inside the machine without using credentials!!!
-
@lakshmana said in Timeout in .sh file:
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
That .sh file should download the data by terminal or by Browser inside the machine without using credentials!!!
Ok, full stop. What happens here is that your web server simply tells you: "Hey, dude, I would love to give you that file, but your f'ing unauthorized, so just leave me alone. I won't give you the file unless you can present me a valid auth".
In other words: HTTP 401 Unauthorized.
So what needs to be done is simple: we need to authenticate against the server. This does not relate to the client side in any way. You will see the same behavior in Internet Explorer, Edge, Chrome, misc. PowerShell cmdlets or whatever else you may use to access that file.
Like SAM said, let us stop talking about client side tools for now. Let us instead discuss very basic authentication mechanisms.
-
@thwr said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
That .sh file should download the data by terminal or by Browser inside the machine without using credentials!!!
Ok, full stop. What happens here is that your web server simply tells you: "Hey, dude, I would love to give you that file, but your f'ing unauthorized, so just leave me alone. I won't give you the file unless you can present me a valid auth".
In other words: HTTP 401 Unauthorized.
So what needs to be done is simple: we need to authenticate against the server. This does not relate to the client side in any way. You will see the same behavior in Internet Explorer, Edge, Chrome, misc. PowerShell cmdlets or whatever else you may use to access that file.
Like SAM said, let us stop talking about client side tools for now. Let us instead discuss very basic authentication mechanisms.
I understood but in Browser the file will download from the cache credentials since the cookies make to download the data Right?????
-
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
That .sh file should download the data by terminal or by Browser inside the machine without using credentials!!!
Ok, full stop. What happens here is that your web server simply tells you: "Hey, dude, I would love to give you that file, but your f'ing unauthorized, so just leave me alone. I won't give you the file unless you can present me a valid auth".
In other words: HTTP 401 Unauthorized.
So what needs to be done is simple: we need to authenticate against the server. This does not relate to the client side in any way. You will see the same behavior in Internet Explorer, Edge, Chrome, misc. PowerShell cmdlets or whatever else you may use to access that file.
Like SAM said, let us stop talking about client side tools for now. Let us instead discuss very basic authentication mechanisms.
I understood but in Browser the file will download from the cache credentials since the cookies make to download the data Right?????
Yep. And it's a very bad idea to rely on something you can't control: a users cached cookies.
-
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
That .sh file should download the data by terminal or by Browser inside the machine without using credentials!!!
Ok, full stop. What happens here is that your web server simply tells you: "Hey, dude, I would love to give you that file, but your f'ing unauthorized, so just leave me alone. I won't give you the file unless you can present me a valid auth".
In other words: HTTP 401 Unauthorized.
So what needs to be done is simple: we need to authenticate against the server. This does not relate to the client side in any way. You will see the same behavior in Internet Explorer, Edge, Chrome, misc. PowerShell cmdlets or whatever else you may use to access that file.
Like SAM said, let us stop talking about client side tools for now. Let us instead discuss very basic authentication mechanisms.
I understood but in Browser the file will download from the cache credentials since the cookies make to download the data Right?????
Right, so that is a reason to avoid that. you need to get the cookie cache and manage it. It's specifically for this kind of reason that cURL is the right tool. I've done exactly this in the past for automation where we needed a login AND a cookie cache and cURL handles this very well.
-
@thwr said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@thwr said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
@scottalanmiller said in Timeout in .sh file:
@lakshmana said in Timeout in .sh file:
@scottalanmiller If i am giving the .sh file to be runned with ny collegues system i cannot share that the executable file since it may lead Infosec Issue
How will that work regardless of the tool used?
That .sh file should download the data by terminal or by Browser inside the machine without using credentials!!!
Ok, full stop. What happens here is that your web server simply tells you: "Hey, dude, I would love to give you that file, but your f'ing unauthorized, so just leave me alone. I won't give you the file unless you can present me a valid auth".
In other words: HTTP 401 Unauthorized.
So what needs to be done is simple: we need to authenticate against the server. This does not relate to the client side in any way. You will see the same behavior in Internet Explorer, Edge, Chrome, misc. PowerShell cmdlets or whatever else you may use to access that file.
Like SAM said, let us stop talking about client side tools for now. Let us instead discuss very basic authentication mechanisms.
I understood but in Browser the file will download from the cache credentials since the cookies make to download the data Right?????
Yep. And it's a very bad idea to rely on something you can't control: a users cached cookies.
This script will not run on users .only on System Administrator machines only
-
That doesn't matter at all. There are only a few factors:
- Do you need any form of authentication - if so, what exactly? A web based form, a HTTP BASIC AUTH (your browser shows a popup for credentials) or something else (some auth protocols can be tricky)
- Can the machine running the script reach the webserver?
- Is there any script language available? not required, just a plus
- Any download tool available? curl, powershell?
You see, it's pretty simple. Most questions are already answered. We now need to know what kind of authentication your webserver requires. Simply start your favorite browser in incognito mode (shift + ctrl + p in Chrome) and navigate to your URL. There are no cookies or caches involved in that moment. Tell us what you see, maybe create a screenshot.
-
@thwr said in Timeout in .sh file:
That doesn't matter at all. There are only a few factors:
- Do you need any form of authentication - if so, what exactly? A web based form, a HTTP BASIC AUTH (your browser shows a popup for credentials) or something else (some auth protocols can be tricky)
- Can the machine running the script reach the webserver?
- Is there any script language available? not required, just a plus
- Any download tool available? curl, powershell?
You see, it's pretty simple. Most questions are already answered. We now need to know what kind of authentication your webserver requires. Simply start your favorite browser in incognito mode (shift + ctrl + p in Chrome) and navigate to your URL. There are no cookies or caches involved in that moment. Tell us what you see, maybe create a screenshot.
As checked the link in the browser by incognito window there is not Login Credentials was asked where the file get downloaded without any issues.
1)As of now i dont need any authorization to download the file
2)The machine which is running is able to reach the Webserver
3)No script language.Just making .sh file to download the files consecutively
4)I am trying the process in Ubuntu and same needs to be worked in Mac but tried the Curl which is creating the file which have the same link inside that file once i click on that link the file will be downloadingI need to download the Files automatically in Downloads Test Folder in the .csv format first
