IT view at the Star Citizen universe - what do you think

[Xist]

Manual entry through in-game chat-bot? Org players are asked from time to time a question about their location followed by 1 or 2 questions about prices for particular commodities. Similarly, trusted???(ha-ha-ha) org players may post to chat their location, commodity, price. Database gets updated.

assumption: chat-bots functionality will be supported

CIG may end up giving us the ability to add mods. Assuming they don't, my first question would be do they allow this sort of automated data scraping.

If they do, the easiest way would probably be an external tool that uses a combination of user input automation and network sniffing and screen scraping.

User walks up to a terminal, clicks their "scan market" button, the tool sends whatever keystrokes are needed to reveal the location and other pertinent info, scrapes the screen for them, then starts systematically paging thru the market and sniffing the network for structured responses.

I'm really hoping CIG open this up with mods tho, it would then be much easier for people and accessible to the masses without significant risk of viruses or other such unpleasantness.

 

[Sirus7264]

CIG may end up giving us the ability to add mods. Assuming they don't, my first question would be do they allow this sort of automated data scraping.

If they do, the easiest way would probably be an external tool that uses a combination of user input automation and network sniffing and screen scraping.

User walks up to a terminal, clicks their "scan market" button, the tool sends whatever keystrokes are needed to reveal the location and other pertinent info, scrapes the screen for them, then starts systematically paging thru the market and sniffing the network for structured responses.

I'm really hoping CIG open this up with mods tho, it would then be much easier for people and accessible to the masses without significant risk of viruses or other such unpleasantness.

As for datamining if they are incomming packets then cig wouldnt really know you are sniffing as long as you are not trying to modify packets that you send back.(undetectable) unless they are running some type of security software which scans your system and informs them you are running or have a particular sniffing application installed. As for screen scraping that would be the best way i'm not sure where technology is with that though or what programs you could use. I'm just worried if they open it up to much this leaves vulnerbilities for hackers to make specialized tools.

 

[Bruce]

In this case the database would have to be embeded into the website either way the data would be outdated. I'm not a big fan of manual input it gets in the way of your day to day gameplay.

Manual input shall be supported but only as a backup strategy in case bots / crawlers are dead due to some rule or interface changes

It has to be automated

I hope that some kind of interface displaying prices of commodities known to the player in the locations known to the player will exist for website, and thus automation would be only question of getting the set of bots that together know about all commodities and all planets/stations/landing zones in the universe

 

[Xist]

I hope that some kind of interface displaying prices of commodities known to the player in the locations known to the player will exist for website, and thus automation would be only question of getting the set of bots that together know about all commodities and all planets/stations/landing zones in the universe

That's my guess/hope as well - that CIG will themselves provide the data via some API that the web site uses.

If they do, it will be trivial to combine that info into some searchable database.

If they don't, they're forcing people to use potentially dangerous ways to collect the info, which is safe neither for CIG, the game or the players. It's better if they provide it themselves.

 

[Vindictive69]

We wouldnt necessarily need to track all the information.. mainly the main commodities that the Org has heavy interest in to start with. I would also think that the larger commodities pricing shouldnt fluctuate all that quickly.

 

[Bruce]

We wouldnt necessarily need to track all the information.. mainly the main commodities that the Org has heavy interest in to start with. I would also think that the larger commodities pricing shouldnt fluctuate all that quickly.

There is no such thing as "too much information" :) ... some deviation of some exotic commodity price in some star system could mean that there is a new factory in this system, and thus some of "common" commodities will be in a need at a high price there soon .. and it would give anyone who has the information some time advantage :)

 

[SweetDee]

My IT career is networking and keeping bad mothers outta my motherlovin' work's network, so when it comes to programming I leave that for another department.

That ATV they had months ago on spinning up VM environments in the cloud...that was cool stuff.

 

[Vindictive69]

There is no such thing as "too much information" :slight_smile: ... some deviation of some exotic commodity price in some star system could mean that there is a new factory in this system, and thus some of "common" commodities will be in a need at a high price there soon .. and it would give anyone who has the information some time advantage :slight_smile:

Cant argue with that. :) But still is always good to have a baseline minimum viable plan to start with and build from. :D

 

[Bruce]

My IT career is networking and keeping bad mothers outta my motherlovin' work's network, so when it comes to programming I leave that for another department.

Oh cmon .. I'm lazy as well, but still - as it was mentioned couple of messages above - sniffing on the end user machine is potentially detectable, and also creating any program like this brings a risk of bad guys taking it and converting into hacking software .. on the other side - it would be piece of cake for any networking guy to provide instructions (or upgrade his own router) to the more or less open version, that would allow install of any of linux sniffers, and careful capture of the packets into cloud storage for future analysis by more programming oriented TESTies :)
// please note - while I have some ideas I usually can't go far away from end user analytical apps .. hence we need a team

 

[Bruce]

Cant argue with that. :slight_smile: But still is always good to have a baseline minimum viable plan to start with and build from. :smile:

I'd go one more step back, and say that better part of the idea of this thread is to understand whether we have a team that could holistically achieve some meaningful baseline (some nerds would call it feasibility study, I'd call it calculation of necessary liquid volume)

 

[Sirus7264]

That's my guess/hope as well - that CIG will themselves provide the data via some API that the web site uses.

If they do, it will be trivial to combine that info into some searchable database.

If they don't, they're forcing people to use potentially dangerous ways to collect the info, which is safe neither for CIG, the game or the players. It's better if they provide it themselves.

Having an api would definitely help but my gut feeling tells me they wont do it to force players to search the verse for the best prices and best places to transfer cargo and goods. It would take away the "exploration" of the market. as for bots at every station that wont happen the verse is to big and that would take a huge ammount of resources.

 

[Vindictive69]

Oh cmon .. I'm lazy as well, but still - as it was mentioned couple of messages above - sniffing on the end user machine is potentially detectable, and also creating any program like this brings a risk of bad guys taking it and converting into hacking software .. on the other side - it would be piece of cake for any networking guy to provide instructions (or upgrade his own router) to the more or less open version, that would allow install of any of linux sniffers, and careful capture of the packets into cloud storage for future analysis by more programming oriented TESTies :slight_smile:
// please note - while I have some ideas I usually can't go far away from end user analytical apps .. hence we need a team

Will need to make sure anything that is done isnt a violation of the ToS... this isnt a FTP game where a ban doesnt really cost people anything.. need to look out for each other cuz we all have too much money invested.. :D.

 

[Bruce]

as for bots at every station that wont happen the verse is to big and that would take a huge ammount of resources.

Definitely not at every station (and even more so - not at every landing point), my assumption (to be verified) would be that WORSE case scenario is 1 bot per faction per system ( i.e. one pirate/smuggler, one trader/hunter , etc. ) .. but I hope for more realistic thing like 1 bot per faction per known to the bot part of universe .. i.e. there shall be a way for the bot to request prices for all known to him materials for all known to him landing zones that are not under blocade etc. this wouldn't remove the need to explore from the game (as one would see only as much as his organization explored), and by the same time should be easy to manage with relatively small resource pool.

 

[Bruce]

Will need to make sure anything that is done isnt a violation of the ToS... this isnt a FTP game where a ban doesnt really cost people anything.. need to look out for each other cuz we all have too much money invested.. :smile:.

I'm 101% with you here (I didn't violate any ToS in this millenia yet, and hope to stay this way). The whole sniffer idea is based on two assumptions 1) packets with market data are not encoded 2) RSI approves collection of such information for informational purposes ... and it has to be verified before we start any coding activities for sure. As I said (and seconded) earlier - personally I expect to have set of APIs that would allow getting this information from out of the game web servers (based on PU knowledge level of the specific account )

 

[AstroSam]

TL;DR
beer.

 

[Bruce]

TL;DR
beer.

AND STEAKS

 

[Thalstan]

One thing i'll say about their current version of 2 step is that it is complete garbage... 2 step when it is done right requires a separate type of physical token either it be a Chipped card, encrypted fingerprint(can be replicated these days but still very hard), or my favorite the good old fashioned code generator(not your phone). If they locked everyone's accounts from gifting and forced anyone who wanted to change current credit card information or gift ships this would solve the biggest issue in SC atm which is hacked accounts. obviously they have been breeched and a lot of old account information is floating around. I already brought up the idea once to them but I think they ignored my suggestion which I expected it is what it is...

As for locations of stuff I'm not sure exactly how you would map that in a database as of right now they don't really have a "Coordinates" type of feature such as X and Y axis here just distance from one jump to the next. I mean it would be possible to do a type of distance type of database. ATM though I don't think a database like that would be really worth it until later. This could be setup pretty simply by making an call send type of SQL database. Also we would have to have someone verify the location and limited access to the database as erroneous information could make the database useless. You could expand out though and use this as an opportunity to make a list of cargo priceing and shop prices at the same time in the same database though.

Two angles and a distance are meaningless unless you know the origin point. What you are referring to is charting a course from your location to a known point. So, to ID a location, what you want is three coordinates in X, Y, and Z, or have it as angle 1, angle 2, and distance from a common reference point (nearest star for items in orbit around a star, the main planet for items that belong to a planetary system. Of course, with orbits, those will change over time as well. This means you need to also know the orbit of the object in the system. If that object is in orbit around the star, it's just one orbit you need to worry about. If it's in orbit around a planet in the system, you need to know the orbit of the planet, and the object around the planet if you want to get to it on one jump. This mean that if you want to get to a particular place, in one jump, you need to know everything about that system (assuming semi-real orbital mechanics are implemented).

Example, you want to get to the Big Benny's broadcast location in Yela, and you are starting from ArcCorp.

First, you need to know where Crusader is in relation to ArcCorp. Then where Yela is in relation to Crusader, then were the Big Benny's vending machine is in it's orbit of Yela. As you can see, it's getting very complex.

Now, a better method would be to use a "beacon" system. Hopefully, they will allow private beacons that are shared only by Orgs or by groups of people. This would allow someone to find a location, drop a beacon that will float along with the location in question (share orbital space) that only responds to a valid code, then does a quick encoded broadcast that allows you to find and then QD to that location directly as long as you have an unobstructed path.

Also, considering the distances involved, trying to find a specific location and QD to it by referencing a distance from a star or even a planet will be very difficult to attempt. Getting within 20 miles of a specific location without a beacon to lead you in when you are talking millions of miles will be close to impossible. The Earth is 93 million miles from the sun (about 150 Million Kilometers) with both values being rounded a bit. Getting within 20 miles will be a the noise level for most distances within a star system.

Now, that's just my 2 credit's worth, but I am not sure the system...as initially stated...will work. Maybe there is something that can work, but it might require some find tuning and a lot of math behind it.

 

[Blind Owl]

TL;DR
beer.

Here I am just clicking "like". SO MUCH READING.

Beer

AND STEAKS

And steaks.

Now we're talking.

 

[Bruce]

First, you need to know where Crusader is in relation to ArcCorp. Then where Yela is in relation to Crusader, then were the Big Benny's vending machine is in it's orbit of Yela. As you can see, it's getting very complex.

Let's go slowly here.
1. Yes, each polar ( 2 angles + distance ) system of coordinates assumes that 0 point is wery well known to everyone who use it, and as you quite correctly mentioned - in the universe it is quite logical to have 0 point for subordinate system known to the super-system . The benefit of having those is quite obvious - as soon as all functions of locations for all 0 points are calculated it become possible to plan missions that involve jumps between systems etc., and we'd definitely require such.

Now, a better method would be to use a "beacon" system. Hopefully, they will allow private beacons that are shared only by Orgs or by groups of people. This would allow someone to find a location, drop a beacon that will float along with the location in question (share orbital space) that only responds to a valid code, then does a quick encoded broadcast that allows you to find and then QD to that location directly as long as you have an unobstructed path.

Beacon system is essencial and (especially in case of tracking of ships, cargo etc. ) sometimes the only way to get exact location of something, BUT with zillions of points of interest in the universe and quite limited beacon lifetime .. attaching beacons to all of them on a regular basis would be a huge task (unfortunately).

Also, considering the distances involved, trying to find a specific location and QD to it by referencing a distance from a star or even a planet will be very difficult to attempt. Getting within 20 miles of a specific location without a beacon to lead you in when you are talking millions of miles will be close to impossible. The Earth is 93 million miles from the sun (about 150 Million Kilometers) with both values being rounded a bit. Getting within 20 miles will be a the noise level for most distances within a star system.

You did a very good explanation of why exactly SC engine is switched to 64 bits from 32 :) ... and yes, w/o mechanism of in-game transport of coordinates having such information will be quite useless :)

Now, that's just my 2 credit's worth, but I am not sure the system...as initially stated...will work. Maybe there is something that can work, but it might require some find tuning and a lot of math behind it.

Oh, and as I keep saying - this thread is first and foremost an attempt to find people who are interested in such topics and design of working systems .. the more the merrier

 

[Sirus7264]

Two angles and a distance are meaningless unless you know the origin point. What you are referring to is charting a course from your location to a known point. So, to ID a location, what you want is three coordinates in X, Y, and Z, or have it as angle 1, angle 2, and distance from a common reference point (nearest star for items in orbit around a star, the main planet for items that belong to a planetary system. Of course, with orbits, those will change over time as well. This means you need to also know the orbit of the object in the system. If that object is in orbit around the star, it's just one orbit you need to worry about. If it's in orbit around a planet in the system, you need to know the orbit of the planet, and the object around the planet if you want to get to it on one jump. This mean that if you want to get to a particular place, in one jump, you need to know everything about that system (assuming semi-real orbital mechanics are implemented).

Example, you want to get to the Big Benny's broadcast location in Yela, and you are starting from ArcCorp.

First, you need to know where Crusader is in relation to ArcCorp. Then where Yela is in relation to Crusader, then were the Big Benny's vending machine is in it's orbit of Yela. As you can see, it's getting very complex.

Now, a better method would be to use a "beacon" system. Hopefully, they will allow private beacons that are shared only by Orgs or by groups of people. This would allow someone to find a location, drop a beacon that will float along with the location in question (share orbital space) that only responds to a valid code, then does a quick encoded broadcast that allows you to find and then QD to that location directly as long as you have an unobstructed path.

Also, considering the distances involved, trying to find a specific location and QD to it by referencing a distance from a star or even a planet will be very difficult to attempt. Getting within 20 miles of a specific location without a beacon to lead you in when you are talking millions of miles will be close to impossible. The Earth is 93 million miles from the sun (about 150 Million Kilometers) with both values being rounded a bit. Getting within 20 miles will be a the noise level for most distances within a star system.

Now, that's just my 2 credit's worth, but I am not sure the system...as initially stated...will work. Maybe there is something that can work, but it might require some find tuning and a lot of math behind it.

yes i know there is a z axis also i left it out because i was being lazy. if you have all 3 coordinates and depending how large they make the square grid of that coordinate that would tell you how possible or impossible it will be to find the object you are looking for. im sure scanners will also play a big roll in this making scout and exploration ships key to this. as for orbital planets that would take extensive work as you are moving massive amounts of polygons a frame at a time pretty much every planet in the universe other than the suns themselves will be huge ships. it would be to much data to process for most computers and cause lag issues. if done that way then planets in that solar system could orbit at different speeds. i dont think cig will do it just to much work and not feasable. you did just make me curious though will each planet spin to show night and day effects properly. If they do that then maybe orbits would be possible. i do say its alot of work.