Has anyone setup an API for an internal application

JaredBusch

@stacksofplates said in Has anyone setup an API for an internal application:

@JaredBusch said in Has anyone setup an API for an internal application:

@stacksofplates said in Has anyone setup an API for an internal application:

So you just want the system to send a text string to a few people? Like through email?

It is invoice data and EFT draft notices
sample from an invoice

Just so I fully understand. They copy that whole CSV or each line would be a separate message to the API?

Preferably, I would like the entire contents of the file to be the API response.

That file structure is the "normal" structure specified by the current solution. We just offered the clients to send the file directly instead of through the EDI provider for those that asked.

Granted the EDI provider manipulates it before sending it back out. All we offered was to send direct.

stacksofplates

@JaredBusch said in Has anyone setup an API for an internal application:

@stacksofplates said in Has anyone setup an API for an internal application:

@JaredBusch said in Has anyone setup an API for an internal application:

@stacksofplates said in Has anyone setup an API for an internal application:

So you just want the system to send a text string to a few people? Like through email?

It is invoice data and EFT draft notices
sample from an invoice

Just so I fully understand. They copy that whole CSV or each line would be a separate message to the API?

Preferably, I would like the entire contents of the file to be the API response.

That file structure is the "normal" structure specified by the current solution. We just offered the clients to send the file directly instead of through the EDI provider for those that asked.

Granted the EDI provider manipulates it before sending it back out. All we offered was to send direct.

Ah ok.

stacksofplates

Honestly I'm not sure what you're going to get doing it that way over just normal SFTP. It would be essentially the same thing just over HTTPS instead. The advantage would come in if the remote ends had a way to consume what you were sending instead of just getting it as a file. I can't speak to EDI because I have no experience with that.

matteo nunziati

@JaredBusch so currently the file is pushed to a server placed at customers' sites?!
You can write some http API but still you need to set the server.
Can't you othetwise keep the sftp server at the source and let customer use any ftp client (even the browser) to download it? Basically this reverse the process snd customers pull the file.

thecreaitvone91

@matteo-nunziati said in Has anyone setup an API for an internal application:

@JaredBusch so currently the file is pushed to a server placed at customers' sites?!
You can write some http API but still you need to set the server.
Can't you othetwise keep the sftp server at the source and let customer use any ftp client (even the browser) to download it? Basically this reverse the process snd customers pull the file.

Winscp offers scripting automation as well so you could send that to them to automate the download to whatever folder. I believe it offers some recording function as well if you don't want to manually script it.

Dashrender

@matteo-nunziati said in Has anyone setup an API for an internal application:

@JaredBusch so currently the file is pushed to a server placed at customers' sites?!
You can write some http API but still you need to set the server.
Can't you othetwise keep the sftp server at the source and let customer use any ftp client (even the browser) to download it? Basically this reverse the process snd customers pull the file.

I believe that Google removed FTP from the browser.
https://www.coywolf.news/productivity/chrome-removing-ftp/

scottalanmiller

@thecreaitvone91 said in Has anyone setup an API for an internal application:

@matteo-nunziati said in Has anyone setup an API for an internal application:

@JaredBusch so currently the file is pushed to a server placed at customers' sites?!
You can write some http API but still you need to set the server.
Can't you othetwise keep the sftp server at the source and let customer use any ftp client (even the browser) to download it? Basically this reverse the process snd customers pull the file.

Winscp offers scripting automation as well so you could send that to them to automate the download to whatever folder. I believe it offers some recording function as well if you don't want to manually script it.

SCP is part of Windows now by default. If your OS is up to date, or if you add it directly as a component, you don't need any third party tools. SSH/SFTP/SCP is there for CMD/PS to use and you can automate that way.

thecreaitvone91

@scottalanmiller said in Has anyone setup an API for an internal application:

@thecreaitvone91 said in Has anyone setup an API for an internal application:

@matteo-nunziati said in Has anyone setup an API for an internal application:

@JaredBusch so currently the file is pushed to a server placed at customers' sites?!
You can write some http API but still you need to set the server.
Can't you othetwise keep the sftp server at the source and let customer use any ftp client (even the browser) to download it? Basically this reverse the process snd customers pull the file.

Winscp offers scripting automation as well so you could send that to them to automate the download to whatever folder. I believe it offers some recording function as well if you don't want to manually script it.

SCP is part of Windows now by default. If your OS is up to date, or if you add it directly as a component, you don't need any third party tools. SSH/SFTP/SCP is there for CMD/PS to use and you can automate that way.

My point was the other side could generate what they needed to do in the GUI of WinSCP rather than having to script it if they didn't know how https://winscp.net/eng/docs/ui_generateurl#script

JaredBusch

@stacksofplates said in Has anyone setup an API for an internal application:

Honestly I'm not sure what you're going to get doing it that way over just normal SFTP. It would be essentially the same thing just over HTTPS instead. The advantage would come in if the remote ends had a way to consume what you were sending instead of just getting it as a file. I can't speak to EDI because I have no experience with that.

Because I do not have an FTP server. The software sends to other people's FTP servers.

But now the customer's customer does not have anything either and they want to pull the data from us.

To me, in the modern world, this means an API to connect and pull their data.

stacksofplates

A question so I can have a better understanding of how the data should be returned. Why the repeating codes in the first line? It looks like that matches up lines, but if it's a CSV I'm not sure why each line has a description for the second column but on another line it's like FD. I'm just trying to understand the structure.

JaredBusch

@stacksofplates said in Has anyone setup an API for an internal application:

A question so I can have a better understanding of how the data should be returned. Why the repeating codes in the first line? It looks like that matches up lines, but if it's a CSV I'm not sure why each line has a description for the second column but on another line it's like FD. I'm just trying to understand the structure.

Are you asking about the first two characters?
IC = Invoice something i don't recall.
IH = Invoice header
IP = Invoice Product
IQ = Invoice Product Message
IX = Invoice Tax
IM = Invoice Message

Not pictured are likely a couple more.

but everyone in the industry knows this format.

My thought was just to dump the text out in a json encoded response or something.

stacksofplates

@JaredBusch said in Has anyone setup an API for an internal application:

@stacksofplates said in Has anyone setup an API for an internal application:

A question so I can have a better understanding of how the data should be returned. Why the repeating codes in the first line? It looks like that matches up lines, but if it's a CSV I'm not sure why each line has a description for the second column but on another line it's like FD. I'm just trying to understand the structure.

Are you asking about the first two characters?
IC = Invoice something i don't recall.
IH = Invoice header
IP = Invoice Product
IQ = Invoice Product Message
IX = Invoice Tax
IM = Invoice Message

Not pictured are likely a couple more.

but everyone in the industry knows this format.

My thought was just to dump the text out in a json encoded response or something.

Yeah you could return that in the JSON body. Are you planning on doing multiple login accounts or just an endpoint per client with a single key?

JaredBusch

@stacksofplates said in Has anyone setup an API for an internal application:

@JaredBusch said in Has anyone setup an API for an internal application:

@stacksofplates said in Has anyone setup an API for an internal application:

A question so I can have a better understanding of how the data should be returned. Why the repeating codes in the first line? It looks like that matches up lines, but if it's a CSV I'm not sure why each line has a description for the second column but on another line it's like FD. I'm just trying to understand the structure.

Are you asking about the first two characters?
IC = Invoice something i don't recall.
IH = Invoice header
IP = Invoice Product
IQ = Invoice Product Message
IX = Invoice Tax
IM = Invoice Message

Not pictured are likely a couple more.

but everyone in the industry knows this format.

My thought was just to dump the text out in a json encoded response or something.

Yeah you could return that in the JSON body. Are you planning on doing multiple login accounts or just an endpoint per client with a single key?

I would assume multiple login accounts are possible, though most people would simply be using automation and only need one.

stacksofplates

@JaredBusch said in Has anyone setup an API for an internal application:

@stacksofplates said in Has anyone setup an API for an internal application:

@JaredBusch said in Has anyone setup an API for an internal application:

@stacksofplates said in Has anyone setup an API for an internal application:

A question so I can have a better understanding of how the data should be returned. Why the repeating codes in the first line? It looks like that matches up lines, but if it's a CSV I'm not sure why each line has a description for the second column but on another line it's like FD. I'm just trying to understand the structure.

Are you asking about the first two characters?
IC = Invoice something i don't recall.
IH = Invoice header
IP = Invoice Product
IQ = Invoice Product Message
IX = Invoice Tax
IM = Invoice Message

Not pictured are likely a couple more.

but everyone in the industry knows this format.

My thought was just to dump the text out in a json encoded response or something.

Yeah you could return that in the JSON body. Are you planning on doing multiple login accounts or just an endpoint per client with a single key?

I would assume multiple login accounts are possible, though most people would simply be using automation and only need one.

yeah it's definitely possible. I was thinking more account per company or server endpoint per company. It's just one is more work which is why I was asking.

So if you're doing the multiple authentication you'll need some middleware to handle the auth on the request so you don't have to call it from each function. Basic auth headers would probably be the easiest to write, however you have to maintain the ids and passwords obviously. You could do OAuth if the clients would be able to do that. Then you'd need to figure out how you want to structure your endpoints. If you just want to give them the latest it could be something like server.com/api/invoice but if they need to access multiple you'd probably want something like server.com/api/invoice/{invoiceId} or somehow by date.

This is kind of bread and butter for a serverless app but if you can't run on a provider you'd need a local server and db. A document based database would probably be the best option here (couchdb, elasticsearch, mongodb, faunadb, etc).

Would you plan on creating the client for your customers or them writing it themselves?

There's going to be a good bit of work in this.

1337

The current setup with the application pushing out information to a sftp server is a better solution than any api in this case.

It protects the application server because it doesn't have any incoming connections from the outside world. It also protects the information because only sftp servers defined in the application will receive the data. So nothing goes anywhere it shouldn't and the application server isn't reachable so it can't be hacked.

So the application dumps the data on a sftp server that the receiving company controls. It could be their own server or it could be a VM somewhere. It's up to them. They could use a $5 vultr instance.

They will then pull the data from that server how they see fit. It's for instance very easy to setup a script that will pull down the files with curl or whatever from the sftp server. And then they can do what they want with the data.

The security responsibility is very easily defined because it's the receiving end that is responsible for that. And sftp is secure but ftp is of course out.

PS. It could also be the sending side that will host and control the sftp servers and manage them. But it only makes sense to me if you are a provider of some kind.

matteo nunziati

@JaredBusch said in Has anyone setup an API for an internal application:

@stacksofplates said in Has anyone setup an API for an internal application:

Honestly I'm not sure what you're going to get doing it that way over just normal SFTP. It would be essentially the same thing just over HTTPS instead. The advantage would come in if the remote ends had a way to consume what you were sending instead of just getting it as a file. I can't speak to EDI because I have no experience with that.

Because I do not have an FTP server. The software sends to other people's FTP servers.

But now the customer's customer does not have anything either and they want to pull the data from us.

To me, in the modern world, this means an API to connect and pull their data.

Well if you reverse the process and go for a pull it is easy to build a small REST API by anything to serve the documents and download them. And the client can be the browser or any simple script with curl if you have to automate via cron/ANY-SCHEDULER-IN-THE-OS.

Btw I remember you had .NET developers. You can build a simple auth'ed app with ASP.NET with VB.NET. It will fit your needs and it seems the easiest road, you just need to define how to store the auth credentials. and if, server side there should be a service listening for someone dropping the new CVS now and then.

matteo nunziati

@Dashrender said in Has anyone setup an API for an internal application:

@matteo-nunziati said in Has anyone setup an API for an internal application:

@JaredBusch so currently the file is pushed to a server placed at customers' sites?!
You can write some http API but still you need to set the server.
Can't you othetwise keep the sftp server at the source and let customer use any ftp client (even the browser) to download it? Basically this reverse the process snd customers pull the file.

I believe that Google removed FTP from the browser.
https://www.coywolf.news/productivity/chrome-removing-ftp/

Yep! still you can use file explorer for sftp if you reverse the ftp architecture. The question is: how much effort has to be put if you have to redo the app in next months?!

stacksofplates

Here's a quick and dirty example with awful authentication but it gets the point across:

package main

import (
	"fmt"
	"log"
	"net/http"
	"time"

	"github.com/gorilla/mux"
)

// index is a fake index page
func index(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "This is the api")
}

// Logger takes and returns an http.Handler.
// It starts a timer and serves up the next handler
// in the chain. When the request is finished it logs the
// request and the amount of time it took.
func Logger(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		start := time.Now()

		next.ServeHTTP(w, r)

		log.Printf(
			"%s %s %s",
			r.Method,
			r.RequestURI,
			time.Since(start),
		)
	})
}

// authenticated is a bad way to check credentials
func authenticated(u, p string) bool {
	if u != "test" || p != "test" {
		return false
	}

	return true
}

// Authentication takes and returns an http.Handler.
// It passes the basic authentication off to authenticated
// and then serves the next handler in the chain if valid 
// credentials are present.
func Authentication(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		user, password, _ := r.BasicAuth()
		if authenticated(user, password) {
			next.ServeHTTP(w, r)
		} else {
			log.Println("request not authenticated")
			http.Error(w, "Unauthorized", http.StatusUnauthorized)
		}

	})
}

// just prints out invoice data
func getInvoices(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusOK)
	w.Write([]byte(`{"invoice_status": "paid"}`))
}

func main() {
	// create new Gorilla mux router
	router := mux.NewRouter()

	// add routes to the router
	router.HandleFunc("/", index).Methods("GET")
	router.HandleFunc("/invoices", getInvoices).Methods("GET")

	// attach our middleware for logging and authentication
	router.Use(Logger)
	router.Use(Authentication)

	//start the server
	log.Fatal(http.ListenAndServe(":8080", router))

}

Here's what the server logs and returns:

matteo nunziati

@stacksofplates yup! One of the few using golang
I've enjoyed it too once!

Unsolved Has anyone setup an API for an internal application