Add contribution guidelines for experimental features (#867)

## What does this PR do?

**Type of change:** ? <!-- Use one of the following: Bug fix, new
feature, new example, new tests, documentation. -->

**Overview:** ?

## Usage
<!-- You can potentially add a usage example below. -->

```python
# Add a code snippet demonstrating how to use this
```

## Testing
<!-- Mention how have you tested your change if applicable. -->

## Before your PR is "*Ready for review*"
<!-- If you haven't finished some of the above items you can still open
`Draft` PR. -->

- **Make sure you read and follow [Contributor
guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md)**
and your commits are signed.
- **Is this change backward compatible?**: Yes/No <!--- If No, explain
why. -->
- **Did you write any new necessary tests?**: Yes/No
- **Did you add or update any necessary documentation?**: Yes/No
- **Did you update
[Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?**:
Yes/No <!--- Only for new features, API changes, critical bug fixes or
bw breaking changes. -->

## Additional Information
<!-- E.g. related issue. -->


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Documentation**
* Added comprehensive guide for experimental optimization technique
development, including recommended structure, testing conventions,
licensing requirements, and graduation path to production.

* **New Features**
* Introduced experimental package with templates and utilities for
implementing research-stage optimization techniques. Includes
configuration framework and example code patterns. Emits stability
warnings to indicate unstable APIs.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Kai Xu <kaix@nvidia.com>

Author: kaix-nv

experimental/README.md

@@ -0,0 +1,135 @@
+# Experimental Optimization Techniques
+
+Experimental optimization algorithms and research prototypes under active development.
+
+## Purpose
+
+For new optimization techniques (quantization, pruning, sparsity, etc.) that are:
+
+- Novel or research-stage algorithms
+- Not yet production-ready
+- May have unstable APIs
+
+**⚠️ Warning**: Experimental features are not guaranteed to work across releases. APIs may change or features may be removed without notice. Use at your own risk.
+
+## Requirements
+
+Each experimental technique must include:
+
+- **README.md** - Explains what the technique does, how to use it, current status, model support, and references
+- **Working code** - Clear, readable implementation
+- **Comprehensive tests** - Good test coverage demonstrating correctness
+- **Detailed documentation** - Clear docs on usage, APIs, and behavior
+- **Example** - Demonstrating usage
+- **Model support list** - Which models/frameworks are supported
+- **Deployment info** - Supported deployment frameworks (TensorRT-LLM, vLLM, SGLang, etc.) and whether custom kernels are required
+- **requirements.txt** - Additional dependencies beyond base modelopt
+- **License headers** - Apache 2.0 headers on all Python files
+
+## Example Structures
+
+Organize your code however makes sense. Here are some examples:
+
+**Simple flat structure:**
+
+```text
+experimental/my_technique/
+├── README.md
+├── requirements.txt
+├── my_technique.py
+├── test_my_technique.py
+└── example.py
+```
+
+**Package structure:**
+
+```text
+experimental/my_technique/
+├── README.md
+├── requirements.txt
+├── my_technique/
+│   ├── __init__.py
+│   ├── core.py
+│   └── config.py
+├── tests/
+│   └── test_core.py
+└── examples/
+    └── example_usage.py
+```
+
+## Quality Standards
+
+Experimental code must meet quality standards:
+
+- Comprehensive test coverage required
+- Clear documentation required
+- Pass all pre-commit checks
+
+## PR Guidelines
+
+Keep PRs focused and reviewable:
+
+- **Split large features**: Break complex techniques into multiple PRs if needed
+- **Reasonable scope**: PRs with tens of thousands of lines are difficult to review
+- **Incremental development**: Consider submitting core functionality first, then enhancements
+- If your technique is large, discuss the implementation plan in an issue first
+
+## Example Documentation Template
+
+Your technique's README.md should include:
+
+```markdown
+# Your Technique Name
+
+Brief description of the optimization technique.
+
+## Model Support
+
+| Model/Framework | Supported | Notes |
+|-----------------|-----------|-------|
+| LLMs (Llama, GPT, etc.) | ✅ | Tested on Llama 3.1 |
+| Diffusion Models | ❌ | Not yet supported |
+| Vision Models | ✅ | Experimental |
+
+## Deployment
+
+| Framework | Supported | Notes |
+|-----------|-----------|-------|
+| TensorRT-LLM | ✅ | Requires custom kernel |
+| vLLM | ❌ | Not yet supported |
+| SGLang | ✅ | Uses standard ops |
+
+## Usage
+
+\`\`\`python
+from experimental.my_technique import my_optimize
+...
+\`\`\`
+
+## Status
+
+Current state: Prototype
+
+Known issues:
+- Issue 1
+- Issue 2
+
+## References
+
+- [Paper](link)
+- [Code repository](link)
+- [Project page](link)
+- [Related work](link)
+```
+
+## Path to Production
+
+When a technique is ready for production (proven effective, stable API, full tests, comprehensive docs), it can be promoted to the main `modelopt` package.
+
+**Contributors**: Open an issue proposing graduation with evidence of effectiveness and stability.
+
+**Users**: If you find an experimental feature valuable, open a GitHub issue requesting promotion to production. User demand is a key signal for production readiness.
+
+## Questions?
+
+Open a GitHub issue with `[experimental]` prefix.

experimental/__init__.py

@@ -0,0 +1,35 @@
+# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Experimental optimization techniques for Model Optimizer.
+
+This package contains experimental and research-stage optimization algorithms
+that are under active development. APIs may change without notice.
+
+Warning:
+    Code in this package is experimental and not covered by semantic versioning.
+    Use at your own risk in production environments.
+"""
+
+import warnings
+
+warnings.warn(
+    "The 'experimental' package contains unstable APIs that may change. "
+    "Use at your own risk in production environments.",
+    FutureWarning,
+    stacklevel=2,
+)
+
+__all__ = []