Skip to content

ci: Add super nightly tests#2855

Draft
ashors1 wants to merge 18 commits into
mainfrom
ashors/super-nightlies
Draft

ci: Add super nightly tests#2855
ashors1 wants to merge 18 commits into
mainfrom
ashors/super-nightlies

Conversation

@ashors1

@ashors1 ashors1 commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

  • Adds 3 nemo gym nightly tests with environments used to post-train nemotron super v3
  • Adds super model nightly tests with nemo-rl's math environment

Note that all configs are simply intended to test functionality and should not be used to test convergence

Depends on:

Issues

List issues that this PR closes (syntax):

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

ashors1 and others added 12 commits June 10, 2026 11:00
Signed-off-by: Anna Shors <ashors@nvidia.com>
Signed-off-by: Anna Shors <ashors@nvidia.com>
Signed-off-by: nemo-ci-bot <nemo-ci-bot@nvidia.com>
Signed-off-by: Anna Shors <ashors@nvidia.com>
Signed-off-by: Anna Shors <ashors@nvidia.com>
Signed-off-by: Anna Shors <ashors@nvidia.com>
Signed-off-by: Anna Shors <ashors@nvidia.com>
Signed-off-by: Anish Mahishi <amahishi@cw-dfw-cs-001-vscode-02.cm.cluster>
Signed-off-by: Anish Mahishi <amahishi@nvidia.com>
Signed-off-by: Anna Shors <ashors@nvidia.com>
Signed-off-by: Anna Shors <ashors@nvidia.com>
Signed-off-by: Anna Shors <ashors@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jun 16, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the Documentation Improvements or additions to documentation label Jun 16, 2026
@ashors1 ashors1 changed the title Ashors/super nightlies ci: Add super nightly tests Jun 16, 2026
ashors1 added 2 commits June 16, 2026 15:16
Signed-off-by: Anna Shors <ashors@nvidia.com>
moe_permute_fusion: true
moe_enable_deepep: false
moe_token_dispatcher_type: "alltoall"
defer_fp32_logits: true # Required when logprob_chunk_size is set

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add mtp training to this recipe?

Suggested change
defer_fp32_logits: true # Required when logprob_chunk_size is set
defer_fp32_logits: true # Required when logprob_chunk_size is set
mtp_loss_scaling_factor: 0.3
mtp_use_repeated_layer: true
mtp_num_layers: 5
mtp_detach_heads: true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants