Scheduling Pattern of Time Triggered Ethernet Based on Reinforcement Learning

Time-triggered Ethernet (TTEthernet or TTE for short) is a deterministic and congestion-free network based on the Ethernet standard. It supports mix-critical real-time applications by providing different message classes. The time-triggered (TT) messages have strict end-to-end delay and accurate jitter requirement, and the rate-constrained (RC) messages have less determinism than TT messages but with bounded end-to-end delay requirement. Traditionally, the scheduling of TT messages makes it free of conflicts for the transmission on physical links, but ignoring RC messages scheduling, so it cannot guarantee the transmission of RC messages with a bounded delay. Therefore, the design of TT schedule becomes the key to TTE network applications within avionics environment. In this paper, we propose an algorithm called RLTS based on reinforcement learning and tree search, to optimize the end-to-end delays of both TT and RC messages. Besides, its computation speed is dozens of times faster than satisfied modularity theory (SMT) with asynchronous method for the calculation of the optimal scheduling table. In the case of a large network with more than 1000 TT and 1000 RC messages, the RLTS method can find a scheduling timetable in 10 seconds, and reduce the worst-case delay of RC messages averagely by 20% compared to the genetic algorithm. Meanwhile, our algorithm has a good generalization performance, in another word, it can quickly adjust itself to satisfy the scheduling with the similar performance as before. By using our method, the scheduling pattern of TTEthernet is further discussed. According to the experimental results, the uniformly distributed slots scheduling pattern, namely the porosity scheduling model which is usually recommended for TTE application, is not always suitable for general situations.