mistral workflow 模板

这两天对之前编写的 mistral workflow 模板进行了修改,将自定义 action 注入用 JS 重写了,下面将依次对改写后的 workflow 模板做一些说明。

下面给出的就算所有写好的 workflow 模板,模板定义了对各种资源的备份和快照创建以及对过期资源的删除操作,并且经过测试应该都是能正常使用的。但需要注意一点的是,在 mistral 中所有资源的创建时间和过期时间都是基于 UTC 零时区的,同样,在使用 cron trigger 进行定时任务的创建时输入的时间也必须是 UTC 零时区的时间,否则在执行时间上就会因我们所处的东八区而有 8 小时的误差。

虚机(server)快照创建及定期删除

创建虚机快照

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
---
version: "2.0"
create_vm_snapshot:
  type: direct
  input:
    - server_id
    - image_name
    - ttl_days
    - ttl_hours
  description: create an image(snapshot) from a server.

  tasks:
    check_vm_ttl:
      description: the values(ttl_days, ttl_hours) should be NUMBER, and at least zero. After this operation, the input values will convert to integers. Otherwise, the next tasks would not be executed!
      action: std.javascript
      input:
        context: <% $ %>
        script: |
          var tag_ttl = 'ERROR';
          var ttl_days = Math.round(Number($.ttl_days));
          var ttl_hours = Math.round(Number($.ttl_hours));
          if(ttl_days >= 0 && ttl_hours >= 0){
            tag_ttl = ttl_days + '_' + ttl_hours;
          }
          else{
            throw new Error('check error!');
          }
          return tag_ttl;
      publish:
        tag_ttl: <% task(check_vm_ttl).result %>
        check_state: SUCCESS
      publish-on-error:
        check_state: ERROR
        reinput_hint: The input values are incorrect and need to input again.
      on-success:
        - create_vm_snapshot: <% $.tag_ttl != ERROR %>

    create_vm_snapshot:
      description: create an image(snapshot) from a server.
      action: nova.servers_create_image server=<% $.server_id %> image_name=<% $.image_name %>
      publish:
        snap_image_id: <% task(create_vm_snapshot).result %>
        create_state: SUCCESS
      publish-on-error:
        create_state: ERROR
      on-success:
        - update_image_workflowtag

    update_image_workflowtag:
      description: create a workflow tag for server.
      action: glance.image_tags_update image_id=<% $.snap_image_id %> tag_value='created_by_mistral'
      publish:
        update_workflowtag_state: SUCCESS
      publish-on-error:
        update_workflowtag_state: ERROR
      on-success:
        - update_image_ttltag

    update_image_ttltag:
      description: create a ttl tag for server.
      action: glance.image_tags_update image_id=<% $.snap_image_id %> tag_value=<% $.tag_ttl %>
      retry:
        delay: 10
        count: 1
      publish:
        update_ttltag_state: SUCCESS
      publish-on-error:
        update_ttltag_state: ERROR
      on-success:
        - wait_for_image

    wait_for_image:
      description: Ensure the vm snapshot was created successfully.
      action: glance.images_get image_id=<% $.snap_image_id %>
      wait-before: 5
      publish:
        tag_value: <% task(wait_for_image).result.tags %>
        wait_state: SUCCESS
      publish-on-error:
        wait_state: ERROR
      retry:
        delay: 10
        count: 5

删除过期虚机快照

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
---
version: '2.0'
judge_expiration_vm_snapshot:
  type: direct
  description: Judge expired snapshots and delete them.

  tasks:
    list_vm_snapshots:
      description: List all vm snapshots created by mistral.
      action: glance.images_list
      publish:
        ids: <% task(list_vm_snapshots).result.where($.tags.len()=2 and $.tags.contains('created_by_mistral')).select($.id) %>
        created_ats: <% task(list_vm_snapshots).result.where($.tags.len()=2 and $.tags.contains('created_by_mistral')).select($.created_at) %>
        ttls: <% task(list_vm_snapshots).result.where($.tags.len()=2 and $.tags.contains('created_by_mistral')).select($.tags[-1]) %>
        list_state: SUCCESS
      publish-on-error:
        list_state: ERROR
      on-success:
        - check_expiration_vm_snapshots

    check_expiration_vm_snapshots:
      description: Find out all expired vm snapshots.
      with-items:
        - image_id in <% $.ids %>
        - created_at in <% $.created_ats %>
        - ttl in <% $.ttls %>
      action: std.javascript
      input:
        context: <% $ %>
        script: |
          var dic = {};
          var ttl = $.ttl.split('_');
          var now_time = new Date();
          var created_time = new Date('<% $.created_at %>');
          var tmp_time = new Date(created_time.setDate(created_time.getDate() + Math.round(Number(ttl[0]))));
          var expire_time = tmp_time.setHours(tmp_time.getHours() + Math.round(Number(ttl[1])));
          dic.enable_delete = (now_time > expire_time);
          dic.now_time = now_time.toISOString();
          dic.expire_time = new Date(expire_time).toISOString();
          dic.resource_id = '<% $.image_id %>';
          return dic;
      retry:
        delay: 5
        count: 10
        continue-on: <% task(check_expiration_vm_snapshots).result.enable_delete = true %>
      publish:
        del_image_ids: <% task(check_expiration_vm_snapshots).result.where($.enable_delete = true).select($.resource_id) %>
        check_state: SUCCESS
      publish-on-error:
        check_state: ERROR
      on-success:
        - delete_vm_snapshots

    delete_vm_snapshots:
      description: Delete all expired vm snapshots.
      with-items: del_image_id in <% $.del_image_ids %>
      action: glance.images_delete image_id=<% $.del_image_id %>
      wait-before: 5
      publish:
        delete_state: SUCCESS
      publish-on-error:
        delete_state: ERROR

创建虚机备份

这一操作因为 nova 的 server backup 的 API 不支持卷启动(volume-backed)的实例的备份,所以根据银联项目需求决定只做虚机实例的快照,不做备份。下面给出的内容只作为参考,不会实际应用。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
---
version: "2.0"
create_vm_backup:
  type: direct
  input:
    - server_id
    - backup_name
    - backup_type: weekly
    - rotation: 1
  description: create a backup from a server.

  tasks:
    create_vm_backup:
      description: create a backup from a server.
      action: nova.servers_backup server=<% $.server_id %> backup_name=<% $.backup_name %> backup_type=<% $.backup_type %> rotation=<% $.rotation %>
      publish:
        server_id: <% task(create_vm_backup).result.id %>
        create_state: SUCCESS
      publish-on-error:
        create_state: ERROR
      on-success:
        - wait_for_backup

    wait_for_backup:
      description: Ensure the vm backup was created successfully.
      action: nova.servers_get server=<% $.server_id %>
      wait-before: 5
      publish:
        wait_state: SUCCESS
      publish-on-error:
        wait_state: ERROR
      retry:
        delay: 10
        count: 30

卷备份和快照创建及定期删除

创建卷备份

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
---
version: "2.0"
create_volume_backup:
  type: direct
  input:
    - volume_id
    - ttl_days
    - ttl_hours
    - force: True
    - incremental: False
  description: create a backup for a volume.

  tasks:
    check_vol_ttl:
      description: the values(ttl_days, ttl_hours) should be NUMBER, and at least zero. After this operation, the input values will convert to integers. Otherwise, the next tasks would not be executed!
      action: std.javascript
      input:
        context: <% $ %>
        script: |
          var tag_ttl = 'ERROR';
          var ttl_days = Math.round(Number($.ttl_days));
          var ttl_hours = Math.round(Number($.ttl_hours));
          if(ttl_days >= 0 && ttl_hours >= 0){
            tag_ttl = ttl_days + '_' + ttl_hours;
          }
          else{
            throw new Error('check error!');
          }
          return tag_ttl;
      publish:
        tag_ttl: <% task(check_vol_ttl).result %>
        check_state: SUCCESS
      publish-on-error:
        check_state: ERROR
        reinput_hint: The input values are incorrect and need to input again.
      on-success:
        - create_vol_backup: <% $.tag_ttl != ERROR %>

    create_vol_backup:
      description: create a backup for a volume.
      action: cinder.backups_create volume_id=<% $.volume_id %> name='created_by_mistral' force=<% $.force %> incremental=<% $.incremental %> description=<% $.tag_ttl %>
      publish:
        backup_id: <% task(create_vol_backup).result.id %>
        create_state: SUCCESS
      publish-on-error:
        create_state: ERROR
      on-success:
        - wait_for_backup

    wait_for_backup:
      description: Ensure the volume backup was created successfully.
      action: cinder.backups_get backup_id=<% $.backup_id %>
      wait-before: 5
      publish:
        wait_state: SUCCESS
      publish-on-error:
        wait_state: ERROR
      retry:
        delay: 10
        count: 5

删除过期卷备份

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
---
version: '2.0'
judge_expiration_vol_backups:
  type: direct
  description: Judge expired backups and delete them.

  tasks:
    list_volume_backups:
      description: List all volume backups created by mistral.
      action: cinder.backups_list
      publish:
        ids: <% task().result.where($.name.len()>0 and $.name='created_by_mistral').id %>
        created_ats: <% task().result.where($.name.len()>0 and $.name='created_by_mistral').select($.created_at) %>
        ttls: <% task().result.where($.name.len()>0 and $.name='created_by_mistral').select($.description) %>
        list_state: SUCCESS
      publish-on-error:
        list_state: ERROR
      on-success:
        - check_expiration_vol_backups

    check_expiration_vol_backups:
      description: Find out all expired volume backups.
      with-items:
        - backup_id in <% $.ids %>
        - created_at in <% $.created_ats %>
        - ttl in <% $.ttls %>
      action: std.javascript
      input:
        context: <% $ %>
        script: |
          var dic = {};
          var ttl = $.ttl.split('_');
          var now_time = new Date();
          var created_time = new Date('<% $.created_at %>');
          var tmp_time = new Date(created_time.setDate(created_time.getDate() + Math.round(Number(ttl[0]))));
          var expire_time = tmp_time.setHours(tmp_time.getHours() + Math.round(Number(ttl[1])));
          dic.enable_delete = (now_time > expire_time);
          dic.now_time = now_time.toISOString();
          dic.expire_time = new Date(expire_time).toISOString();
          dic.resource_id = '<% $.backup_id %>';
          return dic;
      retry:
        delay: 5
        count: 10
        continue-on: <% task(check_expiration_vol_backups).result.enable_delete = true %>
      publish:
        del_backup_ids: <% task(check_expiration_vol_backups).result.where($.enable_delete = true).select($.resource_id) %>
        check_state: SUCCESS
      publish-on-error:
        check_state: ERROR
      on-success:
        - delete_volume_backups

    delete_volume_backups:
      description: Delete all expired volume backups.
      with-items: del_backup_id in <% $.del_backup_ids %>
      action: cinder.backups_delete backup=<% $.del_backup_id %> force=false
      wait-before: 5
      publish:
        delete_state: SUCCESS
      publish-on-error:
        delete_state: ERROR

创建卷快照

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
---
version: "2.0"
create_volume_snapshot:
  type: direct
  input:
    - volume_id
    - snapshot_name
    - ttl_days
    - ttl_hours
    - force: True
    - snapshot_description: Created by mistral
  description: create a snapshot for a volume.

  tasks:
    check_volsnap_ttl:
      description: the values(ttl_days, ttl_hours) should be NUMBER, and at least zero. After this operation, the input values will convert to integers. Otherwise, the next tasks would not be executed!
      action: std.javascript
      input:
        context: <% $ %>
        script: |
          var tag_ttl = 'ERROR';
          var ttl_days = Math.round(Number($.ttl_days));
          var ttl_hours = Math.round(Number($.ttl_hours));
          if(ttl_days >= 0 && ttl_hours >= 0){
            tag_ttl = ttl_days + '_' + ttl_hours;
          }
          else{
            throw new Error('check error!');
          }
          return tag_ttl;
      publish:
        tag_ttl: <% task(check_volsnap_ttl).result %>
        check_state: SUCCESS
      publish-on-error:
        check_state: ERROR
        reinput_hint: The input values are incorrect and need to input again.
      on-success:
        - create_vol_snapshot: <% $.tag_ttl != ERROR %>

    create_vol_snapshot:
      description: create a snapshot for a volume.
      action: cinder.volume_snapshots_create volume_id=<% $.volume_id %> name=<% $.snapshot_name %> force=<% $.force %> description=<% $.snapshot_description %>
      publish:
        vol_snapshot_id: <% task(create_vol_snapshot).result.id %>
        create_state: SUCCESS
      publish-on-error:
        create_state: ERROR
      on-success:
        - set_vol_snapshot_metadata

    set_vol_snapshot_metadata:
      description: set metadata of the volume snapshot.
      action: cinder.volume_snapshots_set_metadata metadata=<% dict(property_ttl => $.tag_ttl, property_tag => "created_by_mistral") %> snapshot=<% $.vol_snapshot_id %>
      publish:
        set_ttltag_state: SUCCESS
      publish-on-error:
        set_ttltag_state: ERROR
      on-success:
        - wait_for_snapshot

    wait_for_snapshot:
      description: Ensure the volume snapshot was created successfully.
      action: cinder.volume_snapshots_get snapshot_id=<% $.vol_snapshot_id %>
      wait-before: 5
      publish:
        wait_state: SUCCESS
      publish-on-error:
        wait_state: ERROR
      retry:
        delay: 10
        count: 5

删除过期卷快照

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
---
version: '2.0'
judge_expiration_vol_snapshot:
  type: direct
  description: Judge expired snapshots and delete them.

  tasks:
    list_volume_snapshots:
      description: List all volume snapshots created by mistral.
      action: cinder.volume_snapshots_list
      publish:
        ids: <% task().result.where($.metadata.len()=2 and $.metadata.property_tag='created_by_mistral').select($.id) %>
        created_ats: <% task().result.where($.metadata.len()=2 and $.metadata.property_tag='created_by_mistral').select($.created_at) %>
        ttls: <% task().result.where($.metadata.len()=2 and $.metadata.property_tag='created_by_mistral').select($.metadata.property_ttl) %>
      on-success:
        - check_expiration_volume_snapshots

    check_expiration_volume_snapshots:
      description: Find out all expired vm snapshots.
      with-items:
        - snapshot_id in <% $.ids %>
        - created_at in <% $.created_ats %>
        - ttl in <% $.ttls %>
      action: std.javascript
      input:
        context: <% $ %>
        script: |
          var dic = {};
          var ttl = $.ttl.split('_');
          var now_time = new Date();
          var created_time = new Date('<% $.created_at %>');
          var tmp_time = new Date(created_time.setDate(created_time.getDate() + Math.round(Number(ttl[0]))));
          var expire_time = tmp_time.setHours(tmp_time.getHours() + Math.round(Number(ttl[1])));
          dic.enable_delete = (now_time > expire_time);
          dic.now_time = now_time.toISOString();
          dic.expire_time = new Date(expire_time).toISOString();
          dic.resource_id = '<% $.snapshot_id %>';
          return dic;
      retry:
        delay: 5
        count: 10
        continue-on: <% task(check_expiration_volume_snapshots).result.enable_delete = true %>
      publish:
        del_vol_snapshot_ids: <% task(check_expiration_volume_snapshots).result.where($.enable_delete = true).select($.resource_id) %>
        check_state: SUCCESS
      publish-on-error:
        check_state: ERROR
      on-success:
        - delete_volume_snapshot

    delete_volume_snapshot:
      description: Delete all expired volume snapshots.
      with-items: del_vol_snapshot_id in <% $.del_vol_snapshot_ids %>
      action: cinder.volume_snapshots_delete snapshot=<% $.del_vol_snapshot_id %> force=false
      wait-before: 5
      publish:
        delete_state: SUCCESS
      publish-on-error:
        delete_state: ERROR

文件共享快照创建及定期删除

创建文件共享快照

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
---
version: "2.0"
create_manila_snapshot:
  type: direct
  input:
    - share_id
    - ttl_days
    - ttl_hours
  description: Create a snapshot of the given share.

  tasks:
    check_manila_ttl:
      description: the values(ttl_days, ttl_hours) should be NUMBER, and at least zero. After this operation, the input values will convert to integers. Otherwise, the next tasks would not be executed!
      action: std.javascript
      input:
        context: <% $ %>
        script: |
          var tag_ttl = 'ERROR';
          var ttl_days = Math.round(Number($.ttl_days));
          var ttl_hours = Math.round(Number($.ttl_hours));
          if(ttl_days >= 0 && ttl_hours >= 0){
            tag_ttl = ttl_days + '_' + ttl_hours;
          }
          else{
            throw new Error('check error!');
          }
          return tag_ttl;
      publish:
        tag_ttl: <% task(check_manila_ttl).result %>
        check_state: SUCCESS
      publish-on-error:
        check_state: ERROR
        reinput_hint: The input values are incorrect and need to input again.
      on-success:
        - create_manila_snapshot: <% $.tag_ttl != ERROR %>

    create_manila_snapshot:
      description: Create a snapshot of the given share.
      action: manila.share_snapshots_create share=<% $.share_id %> name='created_by_mistral' description=<% $.tag_ttl %>
      publish:
        share_snap_id: <% task(create_manila_snapshot).result.id %>
        create_state: SUCCESS
      publish-on-error:
        create_state: ERROR
      on-success:
        - ensure_share_snap

    ensure_share_snap:
      description: Ensure the share snapshot was created successfully.
      action: manila.share_snapshots_get snapshot=<% $.share_snap_id %>
      wait-before: 5
      retry:
        delay: 10
        count: 5
      publish:
        ensure_state: SUCCESS
      publish-on-error:
        ensure_state: ERROR

删除过期文件共享快照

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
---
version: '2.0'
judge_expiration_share_snapshot:
  type: direct
  description: Judge expired share snapshots and delete them.

  tasks:
    list_share_snapshots:
      description: List all share snapshots created by mistral.
      action: manila.share_snapshots_list
      publish:
        ids: <% task(list_share_snapshots).result.where($.name.len()>0 and $.name='created_by_mistral').id %>
        created_ats: <% task(list_share_snapshots).result.where($.name.len()>0 and $.name='created_by_mistral').select($.created_at) %>
        ttls: <% task(list_share_snapshots).result.where($.name.len()>0 and $.name='created_by_mistral').select($.description) %>
        list_state: SUCCESS
      publish-on-error:
        list_state: ERROR
      on-success:
        - check_expiration_sharesnaps

    check_expiration_sharesnaps:
      description: Find out all expired share snapshots.
      with-items:
        - share_snapshot_id in <% $.ids %>
        - created_at in <% $.created_ats %>
        - ttl in <% $.ttls %>
      action: std.javascript
      input:
        context: <% $ %>
        script: |
          var dic = {};
          var ttl = $.ttl.split('_');
          var now_time = new Date();
          var created_time = new Date('<% $.created_at %>');
          var tmp_time = new Date(created_time.setDate(created_time.getDate() + Math.round(Number(ttl[0]))));
          var expire_time = tmp_time.setHours(tmp_time.getHours() + Math.round(Number(ttl[1])));
          dic.enable_delete = (now_time > expire_time);
          dic.now_time = now_time.toISOString();
          dic.expire_time = new Date(expire_time).toISOString();
          dic.resource_id = '<% $.share_snapshot_id %>';
          return dic;
      retry:
        delay: 5
        count: 10
        continue-on: <% task(check_expiration_sharesnaps).result.enable_delete = true %>
      publish:
        del_share_snapshot_ids: <% task(check_expiration_sharesnaps).result.where($.enable_delete = true).select($.resource_id) %>
        check_state: SUCCESS
      publish-on-error:
        check_state: ERROR
      on-success:
        - delete_share_snapshots

    delete_share_snapshots:
      description: Delete all expired share snapshots.
      with-items: del_share_snapshot_id in <% $.del_share_snapshot_ids %>
      action: manila.share_snapshots_delete snapshot=<% $.del_share_snapshot_id %>
      wait-before: 5
      publish:
        delete_state: SUCCESS
      publish-on-error:
        delete_state: ERROR

总结说明

上述每个资源对象(虚机实例,volume,文件共享)的备份或快照都有两个 yaml 文件(一个对应创建,一个对应进行过期检查并删除过期资源)。之所以分开来执行,是因为在执行定时任务创建快照和备份与过期检查的频率可以不一致,当然也可以写在同一个 yaml 文件中(这种方式更简洁,但是删除操作会有延迟现象),那么创建和过期检查的执行频率是一致的。以下总结了这两种方案的特点:

  1. 将创建的动作和检查到期并删除的动作逻辑写在同一个 workflow 中。

这样写的特点是:

  • 当对这个 workflow 创建定时任务时,“创建”和“检查到期并删除”的执行频率是一致的。比如:用户设置定 时任务每隔 5 天执行一次这个 workflow,保留快照时间为 2 天,那么同时“检查到期并删除”也是每隔 5 天执行一次,这时快照只有 2 天的 生存周期,workflow 5 天后再一次被执行时才去“检查到期并删除”,那么这就造成了删除时间的延迟
  • 这种方式只能查询单个的快照是否过期,即当前所创建的快照,过期删除,不能批量查询
  1. 将创建的动作和检查到期并删除的动作逻辑分开写在两个不同的 workflow 中,比如是 wfA 和 wfB

这样写的特点是:

  • 当对这个 wfA 创建定时任务时,它会定时创建快照;wfB 定时”检查到期并删除“,但是这两个 workflow 的 执行频率可以不一致,可自行根据需求来设置
  • 这种方式可以批量查询所有快照是否过期,过期则删除

目前采用第二种方案,可以根据需要进行批量查询,且当设置定时任务时可以将过期检查的频率增大,避免造成删除延迟的现象。

这次使用 JS 来重写时间检测等相关逻辑后就不需要再进行自定义 action 注入了,所以并不会对源码做任何的改动。

注意:因为是使用的 JS 来改写相关逻辑,所以需要用到 pyv8,具体的安装步骤请参考 https://confluence.ustack.com/display/~chenqian/install-pyv8

参考链接