Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8254315: Shenandoah: Concurrent weak reference processing #505

Closed
wants to merge 91 commits into from
Closed
Changes from 2 commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
ac21178
Extend marking bitmap to two bits per HeapWord
Sep 23, 2020
72ccb2e
Add machinery to select different marking strengths for subgraphs
Sep 23, 2020
fc6e187
Rip out old ref-proc stuff, implement rudimentary ShenandoahReference…
Sep 25, 2020
5b50a0d
Implement reference discovery
Sep 25, 2020
2e724a1
Implement concurrent reference processing/enqueueing
Sep 27, 2020
256e403
Consider obj marked-final when already marked strong
Sep 28, 2020
e463af2
Always reset reference-processor thread-locals for all workers
Sep 28, 2020
6d79588
Initialize ShRP::_pending_list and _tail
Sep 28, 2020
60181f5
Add asserts that we only drop references with alive referents
Sep 28, 2020
f41301e
Temporarily disable before-evacuation-verification
Sep 28, 2020
5cb9abd
Process all discovered lists, not just the subset of active worker th…
Sep 28, 2020
5949d47
Prevent copying of thread-local worker data in ShenandoahReferencePro…
Sep 29, 2020
1227b73
Extend ShenandoahVerifier to deal with concurrently-handled j.l.r.Ref…
Sep 30, 2020
fc3309b
Employ relevant barriers in ShenandoahReferenceProcessor
Sep 30, 2020
8ee317c
Extend native-LRB to also be used for Reference.get() intrinsics
Sep 30, 2020
f4f7212
Null-check value after LRB-native and before keep-alive: it may be NULL
Sep 30, 2020
a84f5c2
More trace-logging in reference-processing
Sep 30, 2020
1b4aa18
Inline/flatten bitmap code into ShenandoahMarkBitMap
Oct 1, 2020
637bb25
Implement correct strong and final marking; Fix liveness counting
Oct 1, 2020
ed5cc9f
Use native-LRBs for Reference.get() intrinsics
Oct 5, 2020
d785a52
Install softref policy at init-mark pause, not at conc-mark
Oct 5, 2020
cd9c8f4
Apply LRB when draining ref-proc discovered lists
Oct 5, 2020
1b7ea55
Relax during-evacuation verification to account for Reference referen…
Oct 5, 2020
d7aa041
Perform reference-processing during full-GC and degenerated-GC
Oct 5, 2020
8b1d968
Aarch64 support for concurrent weak references/extended native barriers
Oct 5, 2020
e2f7c71
Relax assert in reference processor to account no LRB in passive mode
Oct 5, 2020
590be2a
Add Oracle copyright header to shenandoahReferenceProcessor.[hc]pp du…
Oct 5, 2020
610cd75
Merge branch 'master' into shenandoah-concurrent-weakrefs
Oct 5, 2020
ef46f90
Reinstate check for ShenandoahSelfFixing that got lost during the merge
Oct 6, 2020
27d3925
Fix null-check after C2 native-LRB
Oct 9, 2020
ccdc963
Add precompiled header to shenandoahMarkBitMap.cpp
Oct 9, 2020
0957efc
Use existing idiom for checking for intanceRefKlass, instead of intro…
Oct 9, 2020
aaf8717
Merge remote-tracking branch 'upstream/master' into shenandoah-concur…
Oct 9, 2020
c9a7c48
Invert weak/native condition in interpreter native-LRB for clarity
Oct 9, 2020
b686b2c
Remove unused is_access_on_jlr_reference() helper method
Oct 9, 2020
ccae1e4
Remove leftovers of precleaning
Oct 9, 2020
70f1b71
Implement abandoning partial discovery. Remove unused methods.
Oct 9, 2020
179002d
Implement reference-processing statistics
Oct 9, 2020
19263bc
Also abandon pending list when abandoning discovered lists
Oct 10, 2020
1661157
Prevent double-discovery of references
Oct 10, 2020
da24231
Merge remote-tracking branch 'upstream/master' into shenandoah-concur…
Oct 10, 2020
83446ec
Merge branch 'master' into shenandoah-concurrent-weakrefs
Oct 12, 2020
7977c88
Add documentation to ShenandoahReferenceProcessor
Oct 12, 2020
070fd83
Merge branch 'master' into shenandoah-concurrent-weakrefs
Oct 12, 2020
34ca499
Don't mark through a Reference that's already been discovered
Oct 12, 2020
4368212
Don't LRB when fetching the referent: we must avoid accidentally maki…
Oct 12, 2020
85fa4d0
Don't allow safepoints when acquiring the Heap_lock for reference enq…
Oct 12, 2020
5e281d5
Carry precise liveness and reachability information in ObjArrayChunkTask
Oct 12, 2020
ee7412e
Remove unnecessary par_is_marked* methods
Oct 12, 2020
93ca92a
Remove wrong assert from ShenandoahReferenceProcessor
Oct 13, 2020
72decd9
Exclude Shenandoah from TestSoftReferencesBehaviorOnOOME.java, it doe…
Oct 13, 2020
2c4b1cb
Explicitely use concurrent vs stw reference processing, don't rely on…
Oct 13, 2020
a3d3aae
Fix 32bit interpreter LRB-native call
Oct 14, 2020
46dc1b7
Add fallback support for new properties in ObjArrayChunkedTask
Oct 14, 2020
f2a9bb6
Change ShenandoahLRBKind to be an enum class instead of plain enum, a…
Oct 21, 2020
6418428
Rename native argument to maybe_narrow_oop for more clarity
Oct 22, 2020
74b3118
Intendation fixes
Oct 27, 2020
2d9764b
Relax verification only for j.l.r.Reference objects
Oct 27, 2020
3a3f1c4
Rename ShenandoahLRBKind -> AccessKind
Oct 27, 2020
c36f745
Merge branch 'master' into shenandoah-concurrent-weakrefs
Oct 27, 2020
02e7ecd
Keep marking closure in ShReferenceProcessor's thread-locals
Oct 28, 2020
05b8c7e
Initialize name and calladdr to make compiler happy about empty defau…
Oct 28, 2020
f9f0dcb
Use template-class instead of template-typename for load_reference_ba…
Oct 28, 2020
787dcb6
Whitespace fixes
Oct 28, 2020
be2c40c
Move back before-evac verification to where it has been
Oct 28, 2020
dbf55c9
Fix copyrights of shenandoahMarkBitMap.*
Oct 28, 2020
a35aaca
Change CWR to CWRF phase timing to avoid clash with conc-weak-roots
Oct 28, 2020
ac75fb1
Fix docs in shenandoahReferenceProcessor.hpp
Oct 28, 2020
5b6fd80
Better encapsulation of the bitmap within ShMarkingContext
Oct 28, 2020
30730b1
Merge branch 'master' into shenandoah-concurrent-weakrefs
Oct 28, 2020
69c81d7
Add ShenandoahWorkerPolicy entry for conc-weak-refs
Oct 28, 2020
0dc1ed8
Consolidate native-LRB invocation
Oct 28, 2020
0561c2a
Merge branch 'master' into shenandoah-concurrent-weakrefs
Oct 28, 2020
072e381
Add missing merge changes of shenandoahTaskQueue.hpp
Oct 28, 2020
803ff8f
Put in comment about API impedence mismatch around interpreter native…
Oct 29, 2020
96f2a3c
Call into native-LRB on unknown oop strenght (i.e. reflection) too
Oct 29, 2020
f2bf4ed
Some more ShMarkTask cleanups
Oct 29, 2020
2587984
Rename mark_final -> mark_weak and several cleanups (by shade)
Oct 29, 2020
f85ab85
Pass marking-strength through chunked arrays
Oct 29, 2020
889389d
Merge branch 'master' into shenandoah-concurrent-weakrefs
Oct 29, 2020
75958ef
Fix merge mistake
Oct 29, 2020
60b39cb
Invert strong/weak in marking tasks and related code
Oct 29, 2020
fc4fb6e
Merge branch 'master' into shenandoah-concurrent-weakrefs
Nov 2, 2020
c09fda9
Adopt ShenandoahReferenceBarrier to recent changes in LRB runtime impl
Nov 2, 2020
4863339
Merge branch 'master' into shenandoah-concurrent-weakrefs
Nov 2, 2020
2c2579d
Invoke interpreter weak-LRB on weak-refs too (fixes merge mistake)
Nov 2, 2020
5e4a8f2
Merge branch 'master' into shenandoah-concurrent-weakrefs
Nov 3, 2020
58dead5
Invert check for access kind (merge mistake)
Nov 3, 2020
9165b6e
AArch64 build fixes
Nov 3, 2020
83799b4
Final touch-ups
Nov 3, 2020
9f367a7
Merge remote-tracking branch 'origin/shenandoah-concurrent-weakrefs' …
Nov 3, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -658,7 +658,7 @@ void ShenandoahBarrierSetAssembler::gen_load_reference_barrier_stub(LIR_Assemble
__ far_call(RuntimeAddress(bs->load_reference_barrier_normal_rt_code_blob()->code_begin()));
break;
case ShenandoahBarrierSet::AccessKind::WEAK:
__ far_call(RuntimeAddress(bs->load_reference_barrier_weakref_rt_code_blob()->code_begin()));
__ far_call(RuntimeAddress(bs->load_reference_barrier_weak_rt_code_blob()->code_begin()));
break;
case ShenandoahBarrierSet::AccessKind::NATIVE:
__ far_call(RuntimeAddress(bs->load_reference_barrier_native_rt_code_blob()->code_begin()));
Original file line number Diff line number Diff line change
@@ -835,7 +835,7 @@ void ShenandoahBarrierSetAssembler::gen_load_reference_barrier_stub(LIR_Assemble
__ call(RuntimeAddress(bs->load_reference_barrier_normal_rt_code_blob()->code_begin()));
break;
case ShenandoahBarrierSet::AccessKind::WEAK:
__ call(RuntimeAddress(bs->load_reference_barrier_weakref_rt_code_blob()->code_begin()));
__ call(RuntimeAddress(bs->load_reference_barrier_weak_rt_code_blob()->code_begin()));
break;
case ShenandoahBarrierSet::AccessKind::NATIVE:
__ call(RuntimeAddress(bs->load_reference_barrier_native_rt_code_blob()->code_begin()));
10 changes: 5 additions & 5 deletions src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.cpp
Original file line number Diff line number Diff line change
@@ -53,7 +53,7 @@ ShenandoahBarrierSetC1::ShenandoahBarrierSetC1() :
_pre_barrier_c1_runtime_code_blob(NULL),
_load_reference_barrier_normal_rt_code_blob(NULL),
_load_reference_barrier_native_rt_code_blob(NULL),
_load_reference_barrier_weakref_rt_code_blob(NULL) {}
_load_reference_barrier_weak_rt_code_blob(NULL) {}

void ShenandoahBarrierSetC1::pre_barrier(LIRGenerator* gen, CodeEmitInfo* info, DecoratorSet decorators, LIR_Opr addr_opr, LIR_Opr pre_val) {
// First we test whether marking is in progress.
@@ -281,9 +281,9 @@ void ShenandoahBarrierSetC1::generate_c1_runtime_stubs(BufferBlob* buffer_blob)
"shenandoah_load_reference_barrier_native_slow",
false, &lrb_native_code_gen_cl);

C1ShenandoahLoadReferenceBarrierCodeGenClosure lrb_weakref_code_gen_cl(ShenandoahBarrierSet::AccessKind::WEAK);
_load_reference_barrier_weakref_rt_code_blob = Runtime1::generate_blob(buffer_blob, -1,
"shenandoah_load_reference_barrier_weakref_slow",
false, &lrb_weakref_code_gen_cl);
C1ShenandoahLoadReferenceBarrierCodeGenClosure lrb_weak_code_gen_cl(ShenandoahBarrierSet::AccessKind::WEAK);
_load_reference_barrier_weak_rt_code_blob = Runtime1::generate_blob(buffer_blob, -1,
"shenandoah_load_reference_barrier_weak_slow",
false, &lrb_weak_code_gen_cl);
}
}
8 changes: 4 additions & 4 deletions src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.hpp
Original file line number Diff line number Diff line change
@@ -192,7 +192,7 @@ class ShenandoahBarrierSetC1 : public BarrierSetC1 {
CodeBlob* _pre_barrier_c1_runtime_code_blob;
CodeBlob* _load_reference_barrier_normal_rt_code_blob;
CodeBlob* _load_reference_barrier_native_rt_code_blob;
CodeBlob* _load_reference_barrier_weakref_rt_code_blob;
CodeBlob* _load_reference_barrier_weak_rt_code_blob;

void pre_barrier(LIRGenerator* gen, CodeEmitInfo* info, DecoratorSet decorators, LIR_Opr addr_opr, LIR_Opr pre_val);

@@ -221,9 +221,9 @@ class ShenandoahBarrierSetC1 : public BarrierSetC1 {
return _load_reference_barrier_native_rt_code_blob;
}

CodeBlob* load_reference_barrier_weakref_rt_code_blob() {
assert(_load_reference_barrier_weakref_rt_code_blob != NULL, "");
return _load_reference_barrier_weakref_rt_code_blob;
CodeBlob* load_reference_barrier_weak_rt_code_blob() {
assert(_load_reference_barrier_weak_rt_code_blob != NULL, "");
return _load_reference_barrier_weak_rt_code_blob;
}
protected:

Original file line number Diff line number Diff line change
@@ -1060,7 +1060,7 @@ Node* ShenandoahBarrierSetC2::ideal_node(PhaseGVN* phase, Node* n, bool can_resh
Node* in1 = n->in(1);
Node* in2 = n->in(2);

// If one input is NULL, then step over the barriers (except LRB native) on the other input
// If one input is NULL, then step over the barriers normal LRB barriers on the other input
if (in1->bottom_type() == TypePtr::NULL_PTR &&
!((in2->Opcode() == Op_ShenandoahLoadReferenceBarrier) &&
((ShenandoahLoadReferenceBarrierNode*)in2)->kind() != ShenandoahBarrierSet::AccessKind::NORMAL)) {
4 changes: 2 additions & 2 deletions src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.hpp
Original file line number Diff line number Diff line change
@@ -50,10 +50,10 @@ class ShenandoahConcurrentMark: public CHeapObj<mtGC> {
inline void do_task(ShenandoahObjToScanQueue* q, T* cl, ShenandoahLiveData* live_data, ShenandoahMarkTask* task);

template <class T>
inline void do_chunked_array_start(ShenandoahObjToScanQueue* q, T* cl, oop array, bool strong);
inline void do_chunked_array_start(ShenandoahObjToScanQueue* q, T* cl, oop array, bool weak);

template <class T>
inline void do_chunked_array(ShenandoahObjToScanQueue* q, T* cl, oop array, int chunk, int pow, bool strong);
inline void do_chunked_array(ShenandoahObjToScanQueue* q, T* cl, oop array, int chunk, int pow, bool weak);

inline void count_liveness(ShenandoahLiveData* live_data, oop obj);