Approximate Dynamic Programming and Reinforcement Learning for Nonlinear Optimal Control of Power Systems

1

• Approximate Dynamic Programming and
  Reinforcement Learning for Nonlinear Optimal
  Control of Power Systems

November 4, 2003
2
Adaptive Critic Design Nonlinear Optimal Control
Model Network
Utility
Plant
(Identifier)
U
Informaton
Function (
)


To learn
the dynamics of plant
Optimal cost-to-go
J
function (
)
Critic Networks

To minimize the value (of derivatives)
Plant
J
of
with respect to the states
Control
Derivatives via BP
Action Network

To find optimal control
Model Network
u
Reinforcement Learning
3
STATCOM Control
4
Simulation Results
100ms SC at PCC, Line Voltage , Generator
Terminal Voltage
5
Optimal control for FACTS devicesInternal
control for static series synchronous compensator
(SSSC)
The simplified schematic of the SSSC (160 MVA,
15KV VL-L)
6
Optimal control for FACTS devicesInternal
control for SSSC (CONVC)
7
Optimal control for FACTS devicesCase study 100
ms three phase short circuit test at
receiving-end (infinite-bus)
8
Optimal control for FACTS devicesExternal
control for series capacitive reactance
compensator (SCRC)
Schematic single-line diagram showing an SCRC
with external controller (160 MVA, 15KV VL-L)
9
Optimal control for FACTS devicesDHP based
external controller (DHPEC)
Schematic single-line diagram showing the DHP
based external controller (DHPEC)
10
Optimal control for FACTS devicesCase study
Step changes XC pu
11
Application in Multi-Machine power system
Large-scale multi-machine power system
12
A UPFC in the POWER SYSTEM
13
Responses of the Generator for a 180 ms 3- phase
Short Circuit at bus 2 at P0.8 p.u Q0.15 p.u
Load angle
Speed response
14
Micro-Machine Research Lab. at the University of
Natal, Durban, South Africa
15
Gen. 1 Trans. Line Impedance Increase
1.01
1
Terminal voltage in pu
0.99
CONV_CONV
0.98
CONV_PSS_CONV
0.97
DHP_CONV
10
10.5
11
11.5
12
12.5
13
13.5
14
14.5
Time in seconds
40
35
Load angle in degrees
CON_CONV
30
CONV_PSS_CONV
25
DHP_CONV
20
10
10.5
11
11.5
12
12.5
13
13.5
14
14.5
15
Time in seconds