.. _optimizer-guide:

=======================
浣跨敤 Optimizer 浼樺寲鍙傛暟
=======================
MegEngine 鐨� :py:mod:`optimizer` 妯″潡涓疄鐜颁簡澶ч噺鐨勪紭鍖栫畻娉曪紝
鍏朵腑 :py:class:`~.Optimizer` 鏄墍鏈変紭鍖栧櫒鐨勬娊璞″熀绫伙紝瑙勫畾浜嗗繀椤绘彁渚涚殑鎺ュ彛銆�
鍚屾椂涓虹敤鎴锋彁渚涗簡鍖呮嫭 :py:class:`~.SGD`, :py:class:`~.Adam` 鍦ㄥ唴鐨勫父瑙佷紭鍖栧櫒瀹炵幇銆�
杩欎簺浼樺寲鍣ㄨ兘澶熷熀浜庡弬鏁扮殑姊害淇℃伅锛屾寜鐓х畻娉曟墍瀹氫箟鐨勭瓥鐣ュ鍙傛暟鎵ц鏇存柊銆�

浠� ``SGD`` 浼樺寲鍣ㄤ负渚嬶紝浼樺寲绁炵粡缃戠粶妯″瀷鍙傛暟鐨勫熀鏈祦绋嬪涓嬶細

.. code-block:: python

   from megengine.autodiff import GradManager
   import megengine.optimizer as optim

   model = MyModel()
   gm = GradManager().attach(model.parameters())
   optimizer = optim.SGD(model.parameters(), lr=0.01)  # lr may vary with different model

   for data, label in dataset:
       with gm:
           pred = model(data)
           loss = loss_fn(pred, label)
           gm.backward(loss)
           optimizer.step().clear_grad()

* 鎴戜滑闇€瑕佹瀯閫犱竴涓紭鍖栧櫒锛屽苟涓斾紶鍏ラ渶瑕佽浼樺寲鐨勫弬鏁� ``Parameter`` 鎴栧叾杩唬锛�
* 閫氳繃鎵ц :py:meth:`~.Optimizer.step` 鏂规硶锛屽弬鏁板皢鍩轰簬姊害淇℃伅琚繘琛屼竴娆′紭鍖栵紱
* 閫氳繃鎵ц :py:meth:`~.Optimizer.clear_grad` 鏂规硶锛屽皢娓呯┖鍙傛暟鐨勬搴︺€�

.. admonition:: 涓轰綍闇€瑕佹墜鍔ㄦ竻绌烘搴︼紵
   :class: warning

   姊害绠＄悊鍣ㄦ墽琛� :py:meth:`~.GradManager.backward` 鏂规硶鏃讹紝
   浼氬皢褰撳墠璁＄畻鎵€寰楀埌鐨勬搴︿互绱姞鐨勫舰寮忕Н绱埌鍘熸湁姊害涓婏紝鑰屼笉鏄洿鎺ュ仛鏇挎崲銆�
   鍥犳瀵逛簬鏂颁竴杞殑姊害璁＄畻锛岄€氬父闇€瑕佸皢涓婁竴杞绠楀緱鍒扮殑姊害淇℃伅娓呯┖銆�
   浣曟椂杩涜姊害娓呯┖鏄敱浜轰负鎺у埗鐨勶紝杩欐牱鍙厑璁哥伒娲昏繘琛屾搴︾殑绱Н銆�

.. _optimizer-state-dict:

Optimizer 鐘舵€佸瓧鍏�
------------------

``Optimizer`` 鏋勯€犲嚱鏁颁腑杩樺彲鎺ュ彈涓€涓惈鏈変紭鍖栧櫒榛樿鍙傛暟鐨勫瓧鍏革紙濡傚惈鏈夊涔犵巼銆佸姩閲忋€佹潈閲嶈“鍑忕郴鏁扮瓑绛夛級锛�
杩欎簺淇℃伅鍙互閫氳繃 :py:meth:`~.Optimizer.state_dict` 鍜� :py:meth:`~.Optimizer.load_state_dict` 鑾峰彇鍜屽姞杞姐€�

瀛愮被鍦ㄥ疄鐜版椂鍙嚜瀹氫箟杩欎簺鍙傛暟锛屽悓鏍蜂互 ``SGD`` 涓轰緥锛�

>>> model = megengine.module.Linear(3, 2)
>>> optimizer = optim.SGD(model.parameters(), lr=1e-3, momentum=0.9, weight_decay=1e-4)
>>> optimizer.state_dict()
{'param_groups': [{'lr': 0.001,
   'momentum': 0.9,
   'weight_decay': 0.0001,
   'params': [0, 1]}],
 'state': {0: {'momentum_buffer': array([0., 0.], dtype=float32)},
  1: {'momentum_buffer': array([[0., 0., 0.],
          [0., 0., 0.]], dtype=float32)}}}

澶у鏁扮殑 Optimizer 鐘舵€佸瓧鍏镐腑浼氬瓨鍌ㄥ弬鏁版搴︾殑缁熻淇℃伅锛堜緥濡傝繍琛屾椂鍧囧€笺€佸弽宸瓑锛夛紝
鍦ㄦ殏鍋�/鎭㈠妯″瀷璁粌鏃讹紝杩欎簺淇℃伅闇€瑕佽淇濆瓨/鍔犺浇锛屼互淇濊瘉鍓嶅悗鐘舵€佺殑涓€鑷存€с€�

.. seealso::

   閫氳繃 :py:meth:`~.Optimizer.load_state_dict` 鎴戜滑鍙互鍔犺浇 ``Optimizer`` 鐘舵€佸瓧鍏革紝甯哥敤浜庢ā鍨嬭缁冭繃绋嬬殑淇濆瓨涓庡姞杞姐€�

   * ``Module`` 涓篃鏈夌敤浜庝繚瀛樺拰鍔犺浇鐨勭姸鎬佸瓧鍏革紝鍙傝€� :ref:`module-guide` 銆�
   * 鍏充簬妯″瀷璁粌杩囩▼涓繚瀛樹笌鍔犺浇鐨勬渶浣冲疄璺碉紝璇峰弬鑰� :ref:`serialization-guide` 銆�

浜嗚В鏇村
--------

.. toctree::
   :maxdepth: 1

   advanced-parameter-optimization