Project Update #8 - GCAP Results

The latest sitrep on PSForever.
User avatar
Chord
Site Admin
Posts: 162
Joined: Wed Oct 14, 2015 4:31 pm
Contact:

Project Update #8 - GCAP Results

Postby Chord » Mon Feb 29, 2016 3:45 am

I've just created a new tool, GCAPy, to help parse the nearly 1 million packets that were submitted.
It's open sourced online and it helped me get some statistics below.

Code: Select all

Packet statistics for 9,813,764 packets

Top 10 Rarest:
 1. 65   0x41        DismountVehicleCargoMsg (1)
 2. 141  0x8d       OutfitMembershipResponse (1)
 3. 184  0xb8         ExperienceAddedMessage (1)
 4. 165  0xa5               WarpgateResponse (3)
 5. 90   0x5a            DelayedPathMountMsg (5)
 6. 101  0x65           AvatarSearchResponse (5)
 7. 228  0xe4           MissionActionMessage (5)
 8. 10    0xa                        HitHint (9)
 9. 34   0x22            ActionCancelMessage (9)
10. 66   0x42   CargoMountPointStatusMessage (12)

Top 10 Most frequent:
 1. 0     0x0                UnknownMessage0 (3548307)
 2. 8     0x8             PlayerStateMessage (2160171)
 3. 44   0x2c     PlanetsideAttributeMessage (968496)
 4. 189  0xbd              UnknownMessage189 (449497)
 5. 27   0x1b            VehicleStateMessage (297619)
 6. 25   0x19            ObjectDeleteMessage (201468)
 7. 97   0x61          ObjectDetectedMessage (184525)
 8. 160  0xa0      BuildingInfoUpdateMessage (152807)
 9. 205  0xcd      DensityLevelUpdateMessage (144753)
10. 80   0x50           TargetingInfoMessage (144126)

Missing 113 packets:
  3 (                    UnknownMessage3)    6 (                    UnknownMessage6)
  7 (                    UnknownMessage7)    9 (                    UnknownMessage9)
 14 (                    MountVehicleMsg)   15 (                 DismountVehicleMsg)
 17 (                   UnknownMessage17)   19 (           CharacterNoRecordMessage)
 21 (                   UnknownMessage21)   26 (                   UnknownMessage26)
 32 (                   UnknownMessage32)   35 (     ActionCancelAcknowledgeMessage)
 43 (                   UnknownMessage43)   45 (                   UnknownMessage45)
 46 (                   UnknownMessage46)   47 (                   UnknownMessage47)
 48 (                   UnknownMessage48)   52 (                  WeaponFireMessage)
 53 (                   UnknownMessage53)   54 (                   UnknownMessage54)
 55 (                   UnknownMessage55)   59 (                   UnknownMessage59)
 60 (                   UnknownMessage60)   64 (               MountVehicleCargoMsg)
 68 (                   UnknownMessage68)   73 (                   UnknownMessage73)
 74 (                SpawnRequestMessage)   76 (                   UnknownMessage76)
 79 (                   UnknownMessage79)   83 (                   UnknownMessage83)
 85 (       DroppodLaunchResponseMessage)   89 (                   UnknownMessage89)
 92 (                   UnknownMessage92)   93 (                   UnknownMessage93)
 94 (                   UnknownMessage94)   95 (                  FavoritesResponse)
 98 (                   UnknownMessage98)  102 (                WeaponJammedMessage)
103 (               LinkDeadAwarenessMsg)  108 (                  UnknownMessage108)
109 (                  UnknownMessage109)  110 (                  UnknownMessage110)
114 (                  UnknownMessage114)  117 (                TrainingZoneMessage)
121 (                       TradeMessage)  122 (                  UnknownMessage122)
124 (                  UnknownMessage124)  125 (                  UnknownMessage125)
126 (                  UnknownMessage126)  131 (                  UnknownMessage131)
133 (             OffshoreVehicleMessage)  136 (                  UnknownMessage136)
137 (                  UnknownMessage137)  139 (                  UnknownMessage139)
140 (                  UnknownMessage140)  142 (                  UnknownMessage142)
147 (               DataChallengeMessage)  148 (                  UnknownMessage148)
151 (                  UnknownMessage151)  154 (                  UnknownMessage154)
156 (                   DebugDrawMessage)  157 (                    SoulMarkMessage)
162 (                  UnknownMessage162)  164 (                  UnknownMessage164)
170 (                  UnknownMessage170)  171 (                  UnknownMessage171)
172 (                  UnknownMessage172)  174 (                    CSAssistMessage)
175 (             CSAssistCommentMessage)  176 (                  UnknownMessage176)
177 (                  UnknownMessage177)  178 (                      VoiceHostInfo)
181 (                  UnknownMessage181)  183 (                  DisconnectMessage)
188 (                           SnoopMsg)  191 (                     ZipLineMessage)
194 (                  UnknownMessage194)  196 (         QuantityDeltaUpdateMessage)
199 (                  UnknownMessage199)  202 (               OutfitBenefitMessage)
204 (            ClockCalibrationMessage)  206 (                    ActOfGodMessage)
208 (                  UnknownMessage208)  210 (              RespawnAMSInfoMessage)
212 (                  UnknownMessage212)  214 (        WarpgateLinkOverrideMessage)
216 (                 ForceEmpireMessage)  218 (                  UnknownMessage218)
219 (                  UnknownMessage219)  221 (                  SquadOrderMessage)
222 (                  UnknownMessage222)  225 (               AudioSequenceMessage)
229 (                  UnknownMessage229)  234 (                RabbitUpdateMessage)
237 (             GameScoreUpdateMessage)  238 (                  UnknownMessage238)
239 (                  UnknownMessage239)  240 (              QueueTimedHelpMessage)
241 (                        MailMessage)  242 (                  UnknownMessage242)
243 (               ClientCheatedMessage)  244 (                  UnknownMessage244)
245 (                  UnknownMessage245)  246 (                  UnknownMessage246)
247 (                  UnknownMessage247)  248 (                  UnknownMessage248)
249 (                  UnknownMessage249)  250 (                  UnknownMessage250)
251 (                  UnknownMessage251)  252 (                  UnknownMessage252)
253 (                  UnknownMessage253)  254 (                  UnknownMessage254)
255 (                  UnknownMessage255)


Some observations from the statistics:
  • There are a lot of missing packets. Most of these have unknown names because a tool I wrote wasn't able to extract all of them from the planetside binary
  • Some things we're never going to be able to capture, such as 'ZipLineMessage' without access to core combat
  • The most frequent packet of 0x00 isn't actually unknown, but it represents a control packet (used for special packets which have their own opcode space). It should be named accordingly

I'm sure there is some more data that can be drawn from this, but it's a good start!
[ PSForever admin and developer ]
PSForever Code | Capturing Packets | Community Discord
User avatar
FateJH
Posts: 95
Joined: Mon Nov 02, 2015 8:37 am

Re: Project Update #8 - GCAP Results

Postby FateJH » Mon Feb 29, 2016 8:11 am

Chord wrote: 2. 141 0x8d OutfitMembershipResponse (1)

An odd quirk of the lower population is that Outfits really haven't mattered for two years now. I still see some TRx run together occasionally and, of course, CN.
243 ( ClientCheatedMessage)

Yes, because people want to admit sending or receiving this packet.
234 ( RabbitUpdateMessage)

Duck season.
The most frequent packet of 0x00 isn't actually unknown, but it represents a control packet (used for special packets which have their own opcode space). It should be named accordingly

Something that I've noticed about the control packet is that it sometimes contains multiple packets and those packets are sometimes detected individually after the original 0x00. Whether that's just a function of the 0x00 being processed, an intentional TCP-ish layer to the UDP, or some other mechanism, is just me speculating.
VS: FateJH, BR 21 CR 0 TR: FJH, BR 18 CR 1 NC: FateJHNC, BR 14 CR 0
User avatar
Chord
Site Admin
Posts: 162
Joined: Wed Oct 14, 2015 4:31 pm
Contact:

Re: Project Update #8 - GCAP Results

Postby Chord » Mon Feb 29, 2016 11:51 am

FateJH wrote:
Chord wrote: 2. 141 0x8d OutfitMembershipResponse (1)

An odd quirk of the lower population is that Outfits really haven't mattered for two years now. I still see some TRx run together occasionally and, of course, CN.
243 ( ClientCheatedMessage)

Yes, because people want to admit sending or receiving this packet.

This will never appear in captures as the capture DLL prevents them from being sent. Didn't want people to get banned!
FateJH wrote:
234 ( RabbitUpdateMessage)

Duck season.

No rabbit games will be played again :(
Should be fairly straight forward to reverse engineer them though.
FateJH wrote:
The most frequent packet of 0x00 isn't actually unknown, but it represents a control packet (used for special packets which have their own opcode space). It should be named accordingly

Something that I've noticed about the control packet is that it sometimes contains multiple packets and those packets are sometimes detected individually after the original 0x00. Whether that's just a function of the 0x00 being processed, an intentional TCP-ish layer to the UDP, or some other mechanism, is just me speculating.

All ControlPackets have a sub-opcode right after 0x00. You are right that one of the jobs is to contain other packets. I call that a MultiPacket. Check out some of the opcodes here: https://github.com/psforever/PSF-LoginS ... code.scala
[ PSForever admin and developer ]
PSForever Code | Capturing Packets | Community Discord
User avatar
Sulferix
Posts: 220
Joined: Sat Nov 28, 2015 8:37 pm

Re: Project Update #8 - GCAP Results

Postby Sulferix » Mon Feb 29, 2016 12:38 pm

WTF?!?! there's like nothing there.....
I guess my impression of how many packets a game should have is false.....

So I suppose pside has less than 1000 different packets? Someone pls clarify. Or are we just missing a whole bunch of packets....
The Ultimate Annihilation of Hackers Is Inevitable
Ryathorz
Posts: 115
Joined: Sat Nov 28, 2015 6:52 am

Re: Project Update #8 - GCAP Results

Postby Ryathorz » Mon Feb 29, 2016 1:29 pm

Sulferix wrote:WTF?!?! there's like nothing there.....
I guess my impression of how many packets a game should have is false.....
Any more and you'd have to send two bytes of ID data instead of one.
User avatar
FateJH
Posts: 95
Joined: Mon Nov 02, 2015 8:37 am

Re: Project Update #8 - GCAP Results

Postby FateJH » Mon Feb 29, 2016 1:59 pm

Sulferix wrote:So I suppose pside has less than 1000 different packets? Someone pls clarify. Or are we just missing a whole bunch of packets....

Ryathorz wrote:Any more and you'd have to send two bytes of ID data instead of one.

If you look at what the GameLogger does, it shows packet data decoded into hexadecimal values. Hexadecimal is a 16-bit number system from 0-15, but 10-15 are represented with A B C D E and F to preserve single-digit-columns. Kinds of data (such as the extremely common UnknownMessage189) always start with the same two decoded symbols (BD) because that identifies what to do with them. BD is 189 in hexadecimal. We have types from UnknownMessage0 to UnknownMessage255, with named message types mixed in in their numerically appropriate order. 0 (decimal) is 00 (hex); and, 255 - the largest integer you can encode with a single byte - is FF. Any higher and you need more bytes.
VS: FateJH, BR 21 CR 0 TR: FJH, BR 18 CR 1 NC: FateJHNC, BR 14 CR 0
User avatar
Chord
Site Admin
Posts: 162
Joined: Wed Oct 14, 2015 4:31 pm
Contact:

Re: Project Update #8 - GCAP Results

Postby Chord » Mon Feb 29, 2016 4:44 pm

FateJH wrote:
Sulferix wrote:So I suppose pside has less than 1000 different packets? Someone pls clarify. Or are we just missing a whole bunch of packets....

Ryathorz wrote:Any more and you'd have to send two bytes of ID data instead of one.

If you look at what the GameLogger does, it shows packet data decoded into hexadecimal values. Hexadecimal is a 16-bit number system from 0-15, but 10-15 are represented with A B C D E and F to preserve single-digit-columns. Kinds of data (such as the extremely common UnknownMessage189) always start with the same two decoded symbols (BD) because that identifies what to do with them. BD is 189 in hexadecimal. We have types from UnknownMessage0 to UnknownMessage255, with named message types mixed in in their numerically appropriate order. 0 (decimal) is 00 (hex); and, 255 - the largest integer you can encode with a single byte - is FF. Any higher and you need more bytes.


Exactly!

Just found out that 0xBD is the PlayerStateMessage. It's sent quite a lot and is probably immediate state info such as speed, yaw, pitch, roll, and position.
[ PSForever admin and developer ]
PSForever Code | Capturing Packets | Community Discord
User avatar
Sulferix
Posts: 220
Joined: Sat Nov 28, 2015 8:37 pm

Re: Project Update #8 - GCAP Results

Postby Sulferix » Tue Mar 01, 2016 12:19 pm

FateJH wrote:
Sulferix wrote:So I suppose pside has less than 1000 different packets? Someone pls clarify. Or are we just missing a whole bunch of packets....

Ryathorz wrote:Any more and you'd have to send two bytes of ID data instead of one.

If you look at what the GameLogger does, it shows packet data decoded into hexadecimal values. Hexadecimal is a 16-bit number system from 0-15, but 10-15 are represented with A B C D E and F to preserve single-digit-columns. Kinds of data (such as the extremely common UnknownMessage189) always start with the same two decoded symbols (BD) because that identifies what to do with them. BD is 189 in hexadecimal. We have types from UnknownMessage0 to UnknownMessage255, with named message types mixed in in their numerically appropriate order. 0 (decimal) is 00 (hex); and, 255 - the largest integer you can encode with a single byte - is FF. Any higher and you need more bytes.


Ya I never looked into the game logger at all really. Thks for clarifying. This should make at least capturing all te packets a hell of a lot easier. Sounds like parseing them is the hard part.
The Ultimate Annihilation of Hackers Is Inevitable

Return to “Project Updates”

Who is online

Users browsing this forum: No registered users and 1 guest