-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Document
The structure of event system is experimental and may be changed in the future.
-
Click
Recordbutton to start recording -
Do anything like clicking mouse or tapping keyboard, which will be recorded
-
Click
Finishbutton to stop recording -
Click
Launchbutton to reproduce the operation recorded in step 2
usage: KeymouseGo.py [-h] [-rt RUNTIMES]
sctipts [sctipts ...]
positional arguments:
sctipts Path for the scripts
optional arguments:
-h, --help show this help message and exit
-rt RUNTIMES, --runtimes RUNTIMES
Run times for the script
Run specific script
> ./KeymouseGo scripts/0314_1452.txt
Run specific script for 3 times
> ./KeymouseGo scripts/0314_1452.txt -rt 3
> ./KeymouseGo scripts/0314_1452.txt --runtimes 3
Assume that the resolution of screen is
1920 * 1080
{
scripts: [
// Press mouse right button at the relative coordinates `(0.05208, 0.1852)`(i.e. absolute coordinates `(100,200)`) after 3000ms
// Note that relative coordinate and absolute coordinate are supported and the recorder uses relative coordinate by default
{type: "event", event_type: "EM", delay: 3000, action_type: "mouse right down", action: ["0.05208%", "0.1852%"]},
// Release mouse right button at the coordinates after 50ms
// The mouse event will execute on the position that the cursor is currently in when the coordinate is set to [-1, -1]
{type: "event", event_type: "EM", delay: 50, action_type: "mouse right up", action: [-1, -1]},
// Press key 'f' after 1000ms
{type: "event", event_type: "EK", delay: 1000, action_type: "key down", action: [70, 'F', 0]},
// Release key 'f' after 50ms
{type: "event", event_type: "EK", delay: 50, action_type: "key up", action: [70, 'F', 0]},
// Press mouse left button at the relative coordinates `(0.2604, 0.4630)`(i.e. absolute coordinates `(500,500)`) after 100ms
{type: "event", event_type: "EM", delay: 100, action_type: "mouse left down", action: ["0.2604%", "0.4630%"]},
// Move mouse to the relative coordinates `(0.2604, 0.4630)`(i.e. absolute coordinates `(500,500)`) after 100ms
{type: "event", event_type: "EM", delay: 100, action_type: "mouse move", action: ["0.2604%", "0.5556%"]},
// Release mouse left button at the relative coordinates `(0.3125, 0.5556)`(i.e. absolute coordinates `(600,600)`) after 100ms
{type: "event", event_type: "EM", delay: 100, action_type: "mouse left up", action: ["0.3125%", "0.5556%"]},
// Input 'Hello world' at current coordinate after 100ms
{type: "event", event_type: "EX", delay: 100, action_type: "input", action: "Hello world"}
]
}The script uses json5 format, with each innermost JSON object representing an event:
Basic format
{
// jsonobject
type: "event",
// The duration between this event and the previous event, measured in milliseconds.
delay: /*<Int>*/ ,v
// Mouse action or keyboard action: `EM` stands for mouse, `EK` stands for keyboard, `EX` stands for other extended action.
event_type: /*<str>*/,
/* Types of Actions:
`mouse left down`: mouse left button pressed, `mouse left up`: mouse left button released,
`mouse right down`: mouse right button pressed, `mouse right up`: mouse right button released,
`mouse middle down`: mouse middle button pressed, `mouse middle up`: mouse middle button released,
`mouse wheel up`: mouse wheel scrolled up, `mouse wheel down`: mouse wheel scrolled down,
`key down`: keyboard key pressed, `key up`: keyboard key released,
`mouse move` mouse moved to position, `input`: typing text.*/
action_type: /*<str>*/,
/* Specific action parameters
Mouse action: consists of two sub-elements namely the horizontal and vertical coordinates [x, y] of the mouse's position on the screen.
When the coordinates are [-1, -1], it indicates that the operation is performed at the current location of the mouse.
Keyboard action: consists of three sub-elements: [key number, key name, extended marker].
Input text action: the content of the text to be entered.*/
action: /*<List>/<str>*/,
// (Optional) Fill in the label of function to be called before execution in the list, multiple functions can be specified, and the corresponding plugins need to be registered in advance.
call: [/* <str>... */],
// (Optional) Label for goto; other types of jsonobjects can also have labels. Please ensure that different jsonobjects are given different labels.
label: /*<str>*/,
// (Optional) Other key-value parameters, can be read by plugin manager during registration.
variable: value
}If you want to apply same action over a series of keyboard and mouse events, this type of event can be used to wrap events. The basic format is:
{
// jsonobject
type: "sequence",
// Key/Mouse event list
events: [/* ... */] ,
// (Optional)The label of function to be called before each execution of event in the list.
// This can simplify the writing for a series of keyboard and mouse events that require adding the same calling function.
attach: [/* <str>... */],
// (Optional) Label for goto; other types of jsonobjects can also have labels. Please ensure that different jsonobjects are given different labels.
label: <str>
}Basic format：
{
// jsonobject
type: "if",
// The program first calls the function specified by `judge`. If it returns True, program will execute the events inside do, and execute events inside else if else.
judge: "registered_function_name",
do: [/* ... */],
else: [/* ... */],
// (Optional) Label for goto; other types of jsonobjects can also have labels. Please ensure that different jsonobjects are given different labels.
label: <str>
}Basic format：
{
// jsonobject
type: "goto",
// the label of jsonobject you wish the process to jump at
tolabel: "label",
// (Optional) Label for goto; other types of jsonobjects can also have labels. Please ensure that different jsonobjects are given different labels.
label: <str>
}It allows the program to run another script during execution. The basic format:
{
// jsonobject
type: "subroutine",
// the other scripts to run sequentially
path: ["subscript_path"],
// (Optional) Label for goto; other types of jsonobjects can also have labels. Please ensure that different jsonobjects are given different labels.
label: <str>
}Simply run the registered function. The basic format：
{
// jsonobject
type: "custom",
// the registered functions to run sequentially
call: [/* <str>... */],
// (Optional) Label for goto; other types of jsonobjects can also have labels. Please ensure that different jsonobjects are given different labels.
label: <str>
}- Please strictly follow the format when making modifications, otherwise it may cause the script to fail to run. It is recommended to back up before making changes.
- The program does not have detection protection for the execution process, please ensure that the execution flow of the script is reasonable.
You can register custom functions by writing plugins. To write a plugin, you need to create a new 'plugins' folder in the program's working directory, and the directory structure of the plugin is like:
/plugins/Myplugin/
- manifest.json5
- .py files
- ...
format of manifest.json5：
// manifest.json5
{
"entry": "Example.py", // the file that implement the plugin interface
"plugin_class": "MyExample", // the class that implement the plugin interface
// (optional) Basic information
"name": "Example",
"description": "This is an example plugin",
"author": "",
"version": "1.0"
}The plugin interface is defined as:
class PluginInterface:
def __init__(self, manifest: Dict):
self.meta = PluginMeta(manifest)
# register function to be called during execution
@abstractmethod
def register_functions(self) -> Dict[str, Callable]:
passregister_functions needs to return a dictionary, where the key corresponds to the function's label. The values is the function to be registered itself. All registered functions will receive a parameter of type JsonObject, and by accessing the content property of that object, the properties of the currently executing object can be obtained.
In the PluginInterface, the content of manifest.json5 is stored in self.meta as a JsonObject, and custom parameters added in the definition file can be accessed through this variable.
The following content achieves the feature of modifying the delay of mouse and keyboard events. Before executing each mouse and keyboard event, the program will call the registered function labeled 'rd' and modify the passed delay parameter, thereby randomly providing an execution delay within a specified range.
// test.json5
{
scripts: [
{
type: "sequence",
events: [
{
type: "event",
delay: "2-8",
event_type: "EM",
action_type: "mouse move",
action: ["0.39427083333333335%","0.45740740740740743%"]
},
{
type: "event",
delay: "1-514",
event_type: "EM",
action_type: "mouse move",
action: ["0.49270833333333336%","0.4398148148148148%"]
}
],
attach: ["rd"]
},
]
}// manifest.json5
{
// 基本信息
"name": "Random Delay",
"description": "Make the format of delay becomes [0-9]+-[0-9]+，i.e. X-Y，where X is the minimal delay and Y is the maximum delay，X,Y are integer mesured in milliseconds",
"author": "Monomux",
"version": "1.0",
"entry": "Random_Delay.py",
"plugin_class": "RandomDelay",
"enabled": true
}Random_Delay.py
from typing import Dict, Callable, List
from Util.Parser import JsonObject
from Plugin.Interface import PluginInterface
from loguru import logger
import re
import random
class RandomDelay(PluginInterface):
def __init__(self, manifest: Dict):
super().__init__(manifest)
def register_functions(self) -> Dict[str, Callable]:
funcs: Dict[str, Callable] = {}
def random_delay(json_object: JsonObject):
delay: str = json_object.content['delay']
# Prevent redundant modification
if type(delay) == str:
delays = re.match('([0-9]+)\-([0-9]+)', delay).groups()
json_object.content['delay'] = random.randint(int(delays[0]), int(delays[1]))
funcs['rd'] = random_delay
return funcsVersion 5.2 removed the speed modification feature, which can now be implemented through sequence events and function calls. The following content implements the feature of modifying the delay of mouse and keyboard events. Before executing each mouse and keyboard event, the program will call the registered function labeled 'rd' and modify the incoming delay parameter.
// test.json5
{
scripts: [
{
type: "sequence",
events: [
{
type: "event",
delay: 50,
event_type: "EM",
action_type: "mouse move",
action: ["0.39427083333333335%","0.45740740740740743%"]
},
{
type: "event",
delay: 50,
event_type: "EM",
action_type: "mouse move",
action: ["0.49270833333333336%","0.4398148148148148%"]
}
],
attach: ["cs"]
},
]
}manifest.json5
{
// 基本信息
"name": "Configure Speed",
"description": "Change the program execution speed, the parameter speed is an integer in the range (0, ∞). Note that the program itself also has execution delay, and the rate will become ineffective beyond a certain point.",
"author": "Monomux",
"version": "1.0",
"entry": "Configure_Speed.py",
"plugin_class": "ConfigureSpeed",
"speed": 0.8 // Speed multiplier, increasing speed when greater than 1.
}ConfigureSpeed.py
from typing import Dict, Callable, Any
from Util.Parser import JsonObject
from Plugin.Interface import PluginInterface
from loguru import logger
class ConfigureSpeed(PluginInterface):
def __init__(self, manifest: Dict):
super().__init__(manifest)
def register_functions(self) -> Dict[str, Callable]:
funcs: Dict[str, Callable] = {}
def change_speed(jsonObject: JsonObject):
delay: int = jsonObject.content['delay']
factor = self.meta.speed
jsonObject.content['delay'] = int(delay / factor)
funcs['cs'] = change_speed
return funcs