Up Top

Serviceability 機能 : JVMTI の処理 : JVMTI 関数の処理 : クラス (Class) : RetransformClasses() 及び RedefineClasses() の処理


概要(Summary)

(See: JVMTI 仕様)

RedefineClasses() 及び RetransformClasses() の処理は VM_RedefineClasses に実装されている.

VM_RedefineClasses 内では,

備考(Notes)

(#TODO 後でまとめる)

    ((cite: hotspot/src/share/vm/prims/jvmtiRedefineClasses.hpp))
    // Introduction:
    //
    // The RedefineClasses() API is used to change the definition of one or
    // more classes. While the API supports redefining more than one class
    // in a single call, in general, the API is discussed in the context of
    // changing the definition of a single current class to a single new
    // class. For clarity, the current class is will always be called
    // "the_class" and the new class will always be called "scratch_class".
    //
    // The name "the_class" is used because there is only one structure
    // that represents a specific class; redefinition does not replace the
    // structure, but instead replaces parts of the structure. The name
    // "scratch_class" is used because the structure that represents the
    // new definition of a specific class is simply used to carry around
    // the parts of the new definition until they are used to replace the
    // appropriate parts in the_class. Once redefinition of a class is
    // complete, scratch_class is thrown away.
    //
    //
    // Implementation Overview:
    //
    // The RedefineClasses() API is mostly a wrapper around the VM op that
    // does the real work. The work is split in varying degrees between
    // doit_prologue(), doit() and doit_epilogue().
    //
    // 1) doit_prologue() is called by the JavaThread on the way to a
    //    safepoint. It does parameter verification and loads scratch_class
    //    which involves:
    //    - parsing the incoming class definition using the_class' class
    //      loader and security context
    //    - linking scratch_class
    //    - merging constant pools and rewriting bytecodes as needed
    //      for the merged constant pool
    //    - verifying the bytecodes in scratch_class
    //    - setting up the constant pool cache and rewriting bytecodes
    //      as needed to use the cache
    //    - finally, scratch_class is compared to the_class to verify
    //      that it is a valid replacement class
    //    - if everything is good, then scratch_class is saved in an
    //      instance field in the VM operation for the doit() call
    //
    //    Note: A JavaThread must do the above work.
    //
    // 2) doit() is called by the VMThread during a safepoint. It installs
    //    the new class definition(s) which involves:
    //    - retrieving the scratch_class from the instance field in the
    //      VM operation
    //    - house keeping (flushing breakpoints and caches, deoptimizing
    //      dependent compiled code)
    //    - replacing parts in the_class with parts from scratch_class
    //    - adding weak reference(s) to track the obsolete but interesting
    //      parts of the_class
    //    - adjusting constant pool caches and vtables in other classes
    //      that refer to methods in the_class. These adjustments use the
    //      SystemDictionary::classes_do() facility which only allows
    //      a helper method to be specified. The interesting parameters
    //      that we would like to pass to the helper method are saved in
    //      static global fields in the VM operation.
    //    - telling the SystemDictionary to notice our changes
    //
    //    Note: the above work must be done by the VMThread to be safe.
    //
    // 3) doit_epilogue() is called by the JavaThread after the VM op
    //    is finished and the safepoint is done. It simply cleans up
    //    memory allocated in doit_prologue() and used in doit().
    //
    //
    // Constant Pool Details:
    //
    // When the_class is redefined, we cannot just replace the constant
    // pool in the_class with the constant pool from scratch_class because
    // that could confuse obsolete methods that may still be running.
    // Instead, the constant pool from the_class, old_cp, is merged with
    // the constant pool from scratch_class, scratch_cp. The resulting
    // constant pool, merge_cp, replaces old_cp in the_class.
    //
    // The key part of any merging algorithm is the entry comparison
    // function so we have to know the types of entries in a constant pool
    // in order to merge two of them together. Constant pools can contain
    // up to 12 different kinds of entries; the JVM_CONSTANT_Unicode entry
    // is not presently used so we only have to worry about the other 11
    // entry types. For the purposes of constant pool merging, it is
    // helpful to know that the 11 entry types fall into 3 different
    // subtypes: "direct", "indirect" and "double-indirect".
    //
    // Direct CP entries contain data and do not contain references to
    // other CP entries. The following are direct CP entries:
    //     JVM_CONSTANT_{Double,Float,Integer,Long,Utf8}
    //
    // Indirect CP entries contain 1 or 2 references to a direct CP entry
    // and no other data. The following are indirect CP entries:
    //     JVM_CONSTANT_{Class,NameAndType,String}
    //
    // Double-indirect CP entries contain two references to indirect CP
    // entries and no other data. The following are double-indirect CP
    // entries:
    //     JVM_CONSTANT_{Fieldref,InterfaceMethodref,Methodref}
    //
    // When comparing entries between two constant pools, the entry types
    // are compared first and if they match, then further comparisons are
    // made depending on the entry subtype. Comparing direct CP entries is
    // simply a matter of comparing the data associated with each entry.
    // Comparing both indirect and double-indirect CP entries requires
    // recursion.
    //
    // Fortunately, the recursive combinations are limited because indirect
    // CP entries can only refer to direct CP entries and double-indirect
    // CP entries can only refer to indirect CP entries. The following is
    // an example illustration of the deepest set of indirections needed to
    // access the data associated with a JVM_CONSTANT_Fieldref entry:
    //
    //     JVM_CONSTANT_Fieldref {
    //         class_index => JVM_CONSTANT_Class {
    //             name_index => JVM_CONSTANT_Utf8 {
    //                 <data-1>
    //             }
    //         }
    //         name_and_type_index => JVM_CONSTANT_NameAndType {
    //             name_index => JVM_CONSTANT_Utf8 {
    //                 <data-2>
    //             }
    //             descriptor_index => JVM_CONSTANT_Utf8 {
    //                 <data-3>
    //             }
    //         }
    //     }
    //
    // The above illustration is not a data structure definition for any
    // computer language. The curly braces ('{' and '}') are meant to
    // delimit the context of the "fields" in the CP entry types shown.
    // Each indirection from the JVM_CONSTANT_Fieldref entry is shown via
    // "=>", e.g., the class_index is used to indirectly reference a
    // JVM_CONSTANT_Class entry where the name_index is used to indirectly
    // reference a JVM_CONSTANT_Utf8 entry which contains the interesting
    // <data-1>. In order to understand a JVM_CONSTANT_Fieldref entry, we
    // have to do a total of 5 indirections just to get to the CP entries
    // that contain the interesting pieces of data and then we have to
    // fetch the three pieces of data. This means we have to do a total of
    // (5 + 3) * 2 == 16 dereferences to compare two JVM_CONSTANT_Fieldref
    // entries.
    //
    // Here is the indirection, data and dereference count for each entry
    // type:
    //
    //    JVM_CONSTANT_Class               1 indir, 1 data, 2 derefs
    //    JVM_CONSTANT_Double              0 indir, 1 data, 1 deref
    //    JVM_CONSTANT_Fieldref            2 indir, 3 data, 8 derefs
    //    JVM_CONSTANT_Float               0 indir, 1 data, 1 deref
    //    JVM_CONSTANT_Integer             0 indir, 1 data, 1 deref
    //    JVM_CONSTANT_InterfaceMethodref  2 indir, 3 data, 8 derefs
    //    JVM_CONSTANT_Long                0 indir, 1 data, 1 deref
    //    JVM_CONSTANT_Methodref           2 indir, 3 data, 8 derefs
    //    JVM_CONSTANT_NameAndType         1 indir, 2 data, 4 derefs
    //    JVM_CONSTANT_String              1 indir, 1 data, 2 derefs
    //    JVM_CONSTANT_Utf8                0 indir, 1 data, 1 deref
    //
    // So different subtypes of CP entries require different amounts of
    // work for a proper comparison.
    //
    // Now that we've talked about the different entry types and how to
    // compare them we need to get back to merging. This is not a merge in
    // the "sort -u" sense or even in the "sort" sense. When we merge two
    // constant pools, we copy all the entries from old_cp to merge_cp,
    // preserving entry order. Next we append all the unique entries from
    // scratch_cp to merge_cp and we track the index changes from the
    // location in scratch_cp to the possibly new location in merge_cp.
    // When we are done, any obsolete code that is still running that
    // uses old_cp should not be able to observe any difference if it
    // were to use merge_cp. As for the new code in scratch_class, it is
    // modified to use the appropriate index values in merge_cp before it
    // is used to replace the code in the_class.
    //
    // There is one small complication in copying the entries from old_cp
    // to merge_cp. Two of the CP entry types are special in that they are
    // lazily resolved. Before explaining the copying complication, we need
    // to digress into CP entry resolution.
    //
    // JVM_CONSTANT_Class and JVM_CONSTANT_String entries are present in
    // the class file, but are not stored in memory as such until they are
    // resolved. The entries are not resolved unless they are used because
    // resolution is expensive. During class file parsing the entries are
    // initially stored in memory as JVM_CONSTANT_ClassIndex and
    // JVM_CONSTANT_StringIndex entries. These special CP entry types
    // indicate that the JVM_CONSTANT_Class and JVM_CONSTANT_String entries
    // have been parsed, but the index values in the entries have not been
    // validated. After the entire constant pool has been parsed, the index
    // values can be validated and then the entries are converted into
    // JVM_CONSTANT_UnresolvedClass and JVM_CONSTANT_UnresolvedString
    // entries. During this conversion process, the UTF8 values that are
    // indirectly referenced by the JVM_CONSTANT_ClassIndex and
    // JVM_CONSTANT_StringIndex entries are changed into Symbol*s and the
    // entries are modified to refer to the Symbol*s. This optimization
    // eliminates one level of indirection for those two CP entry types and
    // gets the entries ready for verification. During class file parsing
    // it is also possible for JVM_CONSTANT_UnresolvedString entries to be
    // resolved into JVM_CONSTANT_String entries. Verification expects to
    // find JVM_CONSTANT_UnresolvedClass and either JVM_CONSTANT_String or
    // JVM_CONSTANT_UnresolvedString entries and not JVM_CONSTANT_Class
    // entries.
    //
    // Now we can get back to the copying complication. When we copy
    // entries from old_cp to merge_cp, we have to revert any
    // JVM_CONSTANT_Class entries to JVM_CONSTANT_UnresolvedClass entries
    // or verification will fail.
    //
    // It is important to explicitly state that the merging algorithm
    // effectively unresolves JVM_CONSTANT_Class entries that were in the
    // old_cp when they are changed into JVM_CONSTANT_UnresolvedClass
    // entries in the merge_cp. This is done both to make verification
    // happy and to avoid adding more brittleness between RedefineClasses
    // and the constant pool cache. By allowing the constant pool cache
    // implementation to (re)resolve JVM_CONSTANT_UnresolvedClass entries
    // into JVM_CONSTANT_Class entries, we avoid having to embed knowledge
    // about those algorithms in RedefineClasses.
    //
    // Appending unique entries from scratch_cp to merge_cp is straight
    // forward for direct CP entries and most indirect CP entries. For the
    // indirect CP entry type JVM_CONSTANT_NameAndType and for the double-
    // indirect CP entry types, the presence of more than one piece of
    // interesting data makes appending the entries more complicated.
    //
    // For the JVM_CONSTANT_{Double,Float,Integer,Long,Utf8} entry types,
    // the entry is simply copied from scratch_cp to the end of merge_cp.
    // If the index in scratch_cp is different than the destination index
    // in merge_cp, then the change in index value is tracked.
    //
    // Note: the above discussion for the direct CP entries also applies
    // to the JVM_CONSTANT_Unresolved{Class,String} entry types.
    //
    // For the JVM_CONSTANT_{Class,String} entry types, since there is only
    // one data element at the end of the recursion, we know that we have
    // either one or two unique entries. If the JVM_CONSTANT_Utf8 entry is
    // unique then it is appended to merge_cp before the current entry.
    // If the JVM_CONSTANT_Utf8 entry is not unique, then the current entry
    // is updated to refer to the duplicate entry in merge_cp before it is
    // appended to merge_cp. Again, any changes in index values are tracked
    // as needed.
    //
    // Note: the above discussion for JVM_CONSTANT_{Class,String} entry
    // types is theoretical. Since those entry types have already been
    // optimized into JVM_CONSTANT_Unresolved{Class,String} entry types,
    // they are handled as direct CP entries.
    //
    // For the JVM_CONSTANT_NameAndType entry type, since there are two
    // data elements at the end of the recursions, we know that we have
    // between one and three unique entries. Any unique JVM_CONSTANT_Utf8
    // entries are appended to merge_cp before the current entry. For any
    // JVM_CONSTANT_Utf8 entries that are not unique, the current entry is
    // updated to refer to the duplicate entry in merge_cp before it is
    // appended to merge_cp. Again, any changes in index values are tracked
    // as needed.
    //
    // For the JVM_CONSTANT_{Fieldref,InterfaceMethodref,Methodref} entry
    // types, since there are two indirect CP entries and three data
    // elements at the end of the recursions, we know that we have between
    // one and six unique entries. See the JVM_CONSTANT_Fieldref diagram
    // above for an example of all six entries. The uniqueness algorithm
    // for the JVM_CONSTANT_Class and JVM_CONSTANT_NameAndType entries is
    // covered above. Any unique entries are appended to merge_cp before
    // the current entry. For any entries that are not unique, the current
    // entry is updated to refer to the duplicate entry in merge_cp before
    // it is appended to merge_cp. Again, any changes in index values are
    // tracked as needed.
    //
    //
    // Other Details:
    //
    // Details for other parts of RedefineClasses need to be written.
    // This is a placeholder section.
    //
    //
    // Open Issues (in no particular order):
    //
    // - How do we serialize the RedefineClasses() API without deadlocking?
    //
    // - SystemDictionary::parse_stream() was called with a NULL protection
    //   domain since the initial version. This has been changed to pass
    //   the_class->protection_domain(). This change has been tested with
    //   all NSK tests and nothing broke, but what will adding it now break
    //   in ways that we don't test?
    //
    // - GenerateOopMap::rewrite_load_or_store() has a comment in its
    //   (indirect) use of the Relocator class that the max instruction
    //   size is 4 bytes. goto_w and jsr_w are 5 bytes and wide/iinc is
    //   6 bytes. Perhaps Relocator only needs a 4 byte buffer to do
    //   what it does to the bytecodes. More investigation is needed.
    //
    // - java.lang.Object methods can be called on arrays. This is
    //   implemented via the arrayKlassOop vtable which we don't
    //   update. For example, if we redefine java.lang.Object.toString(),
    //   then the new version of the method will not be called for array
    //   objects.
    //
    // - How do we know if redefine_single_class() and the guts of
    //   instanceKlass are out of sync? I don't think this can be
    //   automated, but we should probably order the work in
    //   redefine_single_class() to match the order of field
    //   definitions in instanceKlass. We also need to add some
    //   comments about keeping things in sync.
    //
    // - set_new_constant_pool() is huge and we should consider refactoring
    //   it into smaller chunks of work.
    //
    // - The exception table update code in set_new_constant_pool() defines
    //   const values that are also defined in a local context elsewhere.
    //   The same literal values are also used in elsewhere. We need to
    //   coordinate a cleanup of these constants with Runtime.
    //

備考(Notes)

SystemDictionary::parse_stream() は SystemDictionary::resolve_from_stream() とよく似た関数だが, 補助的なデータ構造の更新を一切行わない点が違う模様.

    ((cite: hotspot/src/share/vm/classfile/systemDictionary.cpp))
    // Note: this method is much like resolve_from_stream, but
    // updates no supplemental data structures.
    // TODO consolidate the two methods with a helper routine?
    klassOop SystemDictionary::parse_stream(Symbol* class_name,

備考(Notes)

なお, TraceRedefineClasses というコマンドラインオプションで RedefineClasses 機能に関するトレース出力レベルを指定することができる. 指定する値は以下を参照.

    ((cite: hotspot/src/share/vm/prims/jvmtiRedefineClassesTrace.hpp))
    // RedefineClasses tracing support via the TraceRedefineClasses
    // option. A bit is assigned to each group of trace messages.
    // Groups of messages are individually selectable. We have to use
    // decimal values on the command line since the command option
    // parsing logic doesn't like non-decimal numerics. The HEX values
    // are used in the actual RC_TRACE() calls for sanity. To achieve
    // the old cumulative behavior, pick the level after the one in
    // which you are interested and subtract one, e.g., 33554431 will
    // print every tracing message.
    //
    //    0x00000000 |          0 - default; no tracing messages
    //    0x00000001 |          1 - name each target class before loading, after
    //                              loading and after redefinition is completed
    //    0x00000002 |          2 - print info if parsing, linking or
    //                              verification throws an exception
    //    0x00000004 |          4 - print timer info for the VM operation
    //    0x00000008 |          8 - print subclass counter updates
    //    0x00000010 |         16 - unused
    //    0x00000020 |         32 - unused
    //    0x00000040 |         64 - unused
    //    0x00000080 |        128 - unused
    //    0x00000100 |        256 - previous class weak reference addition
    //    0x00000200 |        512 - previous class weak reference mgmt during
    //                              class unloading checks (GC)
    //    0x00000400 |       1024 - previous class weak reference mgmt during
    //                              add previous ops (GC)
    //    0x00000800 |       2048 - previous class breakpoint mgmt
    //    0x00001000 |       4096 - detect calls to obsolete methods
    //    0x00002000 |       8192 - fail a guarantee() in addition to detection
    //    0x00004000 |      16384 - unused
    //    0x00008000 |      32768 - old/new method matching/add/delete
    //    0x00010000 |      65536 - impl details: CP size info
    //    0x00020000 |     131072 - impl details: CP merge pass info
    //    0x00040000 |     262144 - impl details: CP index maps
    //    0x00080000 |     524288 - impl details: modified CP index values
    //    0x00100000 |    1048576 - impl details: vtable updates
    //    0x00200000 |    2097152 - impl details: itable updates
    //    0x00400000 |    4194304 - impl details: constant pool cache updates
    //    0x00800000 |    8388608 - impl details: methodComparator info
    //    0x01000000 |   16777216 - impl details: nmethod evolution info
    //    0x02000000 |   33554432 - impl details: annotation updates
    //    0x04000000 |   67108864 - impl details: StackMapTable updates
    //    0x08000000 |  134217728 - impl details: OopMapCache updates
    //    0x10000000 |  268435456 - unused
    //    0x20000000 |  536870912 - unused
    //    0x40000000 | 1073741824 - unused
    //    0x80000000 | 2147483648 - unused
    //
    // Note: The ResourceMark is to cleanup resource allocated args.
    //   The "while (0)" is so we can use semi-colon at end of RC_TRACE().

備考(Notes)

_has_redefined_a_class は, #TODO

    ((cite: hotspot/src/share/vm/prims/jvmtiExport.hpp))
      // The RedefineClasses() API breaks some invariants in the "regular"
      // system. For example, there are sanity checks when GC'ing nmethods
      // that require the containing class to be unloading. However, when a
      // method is redefined, the old method and nmethod can become GC'able
      // without the containing class unloading. The state of becoming
      // GC'able can be asynchronous to the RedefineClasses() call since
      // the old method may still be running and cannot be GC'ed until
      // after all old invocations have finished. Additionally, a method
      // that has not been redefined may have an nmethod that depends on
      // the redefined method. The dependent nmethod will get deopted in
      // this case and may also be GC'able without the containing class
      // being unloaded.
      //
      // This flag indicates whether RedefineClasses() has ever redefined
      // one or more classes during the lifetime of the VM. The flag should
      // only be set by the friend class and can be queried by other sub
      // systems as needed to relax invariant checks.
      static bool _has_redefined_a_class;
      friend class VM_RedefineClasses;
      inline static void set_has_redefined_a_class() {
        _has_redefined_a_class = true;
      }

処理の流れ (概要)(Execution Flows : Summary)

RedefineClasses() の処理

JvmtiEnv::RedefineClasses()
-> VMThread::execute()
   -> (略) (See: here for details)
      -> VM_RedefineClasses::doit_prologue()
         -> VM_RedefineClasses::load_new_class_versions()
            -> SystemDictionary::parse_stream()
               -> ClassFileParser::parseClassFile()
                  -> 
            -> Verifier::verify()
            -> Rewriter::rewrite()
            -> Rewriter::relocate_and_link()
            ...#TODO
      -> VM_RedefineClasses::doit()
         -> redefine するクラス数だけ redefine_single_class() を呼び出す
            VM_RedefineClasses::redefine_single_class()
         -> SystemDictionary::notice_modification();
         -> JvmtiExport::set_has_redefined_a_class();
      -> VM_RedefineClasses::doit_epilogue()

RetransformClasses() の処理

JvmtiEnv::RetransformClasses()
-> JvmtiClassFileReconstituter::JvmtiClassFileReconstituter()
   -> JvmtiClassFileReconstituter::write_class_file_format()
-> JvmtiClassFileReconstituter::class_file_size()
-> JvmtiClassFileReconstituter::class_file_bytes()
-> VMThread::execute()
   -> (略) (See: here for details)
      -> VM_RedefineClasses::doit_prologue()
         -> (同上)
      -> VM_RedefineClasses::doit()
         -> (同上)
      -> VM_RedefineClasses::doit_epilogue()

処理の流れ (詳細)(Execution Flows : Details)

JvmtiEnv::RedefineClasses()

See: here for details

VM_RedefineClasses::doit_prologue()

See: here for details

VM_RedefineClasses::load_new_class_versions()

(#Under Construction) See: here for details

VM_RedefineClasses::doit()

See: here for details

VM_RedefineClasses::redefine_single_class()

(#Under Construction) See: here for details

VM_RedefineClasses::check_methods_and_mark_as_obsolete()

(#Under Construction) See: here for details

VM_RedefineClasses::doit_epilogue()

See: here for details

JvmtiEnv::RetransformClasses()

See: here for details

JvmtiClassFileReconstituter::JvmtiClassFileReconstituter()

See: here for details

JvmtiClassFileReconstituter::class_file_size()

See: here for details

JvmtiClassFileReconstituter::class_file_bytes()

See: here for details

JvmtiClassFileReconstituter::write_class_file_format()

See: here for details


This document is available under the GNU GENERAL PUBLIC LICENSE Version 2.