Skip to content

Commit 240a86c

Browse files
ondrejmularondrejmular
committed
testing: adds mitmproxy (https://github.com/saironiq/mitmproxy)
1 parent fee487e commit 240a86c

27 files changed

+4007
-0
lines changed

‎mitm/mitmproxy-0.1/INTERNAL.md

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
MITMPROXY LIBRARY INTERNALS
2+
===========================
3+
4+
5+
Introduction
6+
------------
7+
The `mitmproxy` library consists of 2 modules: `mitmproxy.py` and `sshdebug.py`.
8+
Module `mitmproxy.py` contains implementation of proxy and replay servers
9+
for supported network protocols. Module `sshdebug.py` serves for more human-friendly
10+
debugging output of unencrypted SSH messages.
11+
12+
The library is built on the Python Twisted network framework - see the project homepage
13+
at [twistedmatrix.com](http://twistedmatrix.com/).
14+
15+
16+
Supported network protocols
17+
---------------------------
18+
* Telnet
19+
* HTTP
20+
* HTTPS/SSL
21+
* SSH
22+
* SNMP
23+
24+
25+
MITM proxy server in general
26+
----------------------------
27+
```
28+
+----------------------MITM PROXY---------------------+
29+
| +--------------+ +-------+ +--------------+ |
30+
| | (receive) | <<< | Queue | <<< | (transmit) | |
31+
+--------+ | | | +-------+ | | | +--------+
32+
| Client | <--> | | Proxy Server | | Proxy Client | | <--> | Server |
33+
+--------+ | | | +-------+ | | | +--------+
34+
| | (transmit) | >>> | Queue | >>> | (receive) | |
35+
| +--------------+ +-------+ +--------------+ |
36+
+-----------------------------------------------------+
37+
```
38+
39+
As you can see in the above diagram, the MITM proxy has 2 componets - proxy server
40+
and proxy client. These components communicate internally via deferred queues
41+
and all of the communication is logged for later use by the replay server.
42+
For details about deferred queues RTFM at [defer-intro](http://twistedmatrix.com/documents/current/core/howto/defer-intro.html).
43+
44+
More info about TCP/UDP clients/servers in Twisted is available on the Twisted core [doc pages](http://twistedmatrix.com/documents/current/core/howto/index.html).
45+
Most of the proxies are quite similar - one exception is SSH because it has multiple layers.
46+
MITM SSH proxy and replay servers are using implementation of SSHv2 protocol
47+
from `twisted.conch.ssh` package. There are some examples of SSH clients and
48+
servers on twisted conch documentation pages:
49+
[conch doc](http://twistedmatrix.com/documents/current/conch/index.html).
50+
Another `twisted.conch.ssh` howto can be found here:
51+
[ticket-5474](http://twistedmatrix.com/trac/ticket/5474)
52+
and
53+
[ticket-6001](http://twistedmatrix.com/trac/ticket/6001).
54+
55+
56+
Notes on SSH
57+
------------
58+
### SSH keypairs
59+
SSH proxy/replay requires SSH keypairs even when they are not being used,
60+
so you should generate them with either `mitmkeygen` or by hand (and specify
61+
their location on command-line if you put them in non-default location).
62+
63+
### Communication
64+
The communication between proxy components starts during authentication.
65+
Proxy server first negotiates authentication method with the client
66+
and then forwards it to the proxy client, which in turn tries authenticating
67+
against the real server. Then the proxy client informs proxy server about
68+
the authentication result. After SSH transport layer connection is established,
69+
the proxy simply forwards (decrypts and re-encrypts) connection layer packets.
70+
The forwarding is done by the `mitmproxy.ProxySSHConnection` class which is a
71+
subclass of `ssh.connection.SSHConnection`. The `mitmproxy.ProxySSHConnection` class simply
72+
overrides the `packetReceived` method which puts received packets into a queue and logs
73+
the SSH channel data (`SSH_MSG_CHANNEL_DATA` message type), which is assumed to be
74+
the interactive shell channel.
75+
76+
### Authentication
77+
The client connects to proxy, the proxy in turn creates a connection
78+
to real server and forwards the SSH banner to the client. The client then chooses
79+
an authentication method to use and authenticates against the _proxy_.
80+
In case of password auth the proxy simply forwards it to the real server.
81+
With public key auth, however, the proxy needs to have the corresponding private
82+
key to be able to authenticate client - so the proxy has its own keypair
83+
to use for client connection. This also means that the proxy has to have the client's
84+
keypair (or any other keypair that is accepted as valid for the given user on the real server).
85+
Thus for the proxy to work with pubkey auth you need to add the public key of _proxy_ to
86+
the list of allowed keys for the give user at the given real server (usually ~/.ssh/authorized_keys).
87+
88+
89+
The proxy server uses Twisted's [Pluggable Authentication](http://twistedmatrix.com/documents/current/core/howto/cred.html) system.
90+
Proxy authentication is implemented in these classes:
91+
* ProxySSHUserAuthServer
92+
* ProxySSHUserAuthClient
93+
* SSHCredentialsChecker
94+
95+
96+
Proxy server side is implemented mostly in the `SSHCredentialsChecker` class.
97+
The `ProxySSHUserAuthServer` is a hack to be able to properly end the communication.
98+
The authentication result is evaluted in callback method `SSHCredentialsChecker.is_auth_succes`.
99+
There are three possible results:
100+
* auth succeeded
101+
* auth failed, more auth methods available - try another one
102+
* auth failed, no more auth methods - disconnect
103+
104+
105+
Proxy client auth is implemented in `ProxySSHUserAuthClient`.
106+
It sends/receives information to/from proxy server through deferred queues.
107+
After successful auth the ssh-connection service is started.
108+
109+
110+
There are some issues with proxy auth:
111+
* proxy and real client auth method attempt order must be the same
112+
* real server might support less auth methods than proxy
113+
114+
First issue is solved by sending the name of current auth method used by client to proxy.
115+
Second issue is solved by pretending method failure and waiting for another auth method.
116+
117+
The proxy tries to be as transparent as possible - everything depends only on server and client configuration
118+
(eg. the number of allowed password auth attemps) - well, at least it *should*. ;)
119+
120+
121+
### SSH Replay server
122+
SSH replay server always successfully authenticates client on the first authentication method attempt.
123+
The replay server is implemented in `mitmproxy.SSHReplayServerProtocol` and it is connected to the SSH service in
124+
`mitmproxy.ReplayAvatar` class.
125+
126+
### SSH proxy drawbacks
127+
The proxy only supports logging of SSH sessions with _only one_ open channel.
128+
It does not support other SSH features like port forwarding.
129+
130+
131+
Notes about SNMP proxy and replay server
132+
----------------------------------------
133+
SNMP proxy is ordinary UDP server that intercepts and forwards UDP packets.
134+
Because of UDP protocol we do not know when the communication ends.
135+
You can save the PID of SNMP proxy process and when the client ends you can terminate the proxy.
136+
SNMP replay reads communication from log and compares received packets with expected packets.
137+
It copes with different request IDs in packets.
138+
There are functions for extracting and replacing the request-id from/in packets - `snmp_extract_request_id` and `snmp_replace_request_id`.
139+
SNMP replay server works only with UDP.
140+

0 commit comments

Comments
 (0)