Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8263018: Improve API for lifecycle of native resources #466

Closed
Closed
Changes from 3 commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
85b1246
Initial (hacky) pass.
mcimadamore Jan 14, 2021
0d52da8
Cleanup VaList implementation
mcimadamore Jan 14, 2021
911d739
Fix benchmarks
mcimadamore Jan 14, 2021
9c8799a
All tests pass
mcimadamore Jan 15, 2021
b0790bd
Cleanup scope logic - pull resource list into its own abstraction
mcimadamore Jan 15, 2021
5628993
Add NativeAllocator API.
mcimadamore Jan 18, 2021
f2f6946
Add initial implementation of ResourceScope::fork
mcimadamore Jan 18, 2021
87cfd70
Improve implementation of NativeScope
mcimadamore Jan 18, 2021
e71a193
Remove access modes
mcimadamore Jan 18, 2021
a46a480
Add biggie test for ResourceScope::fork
mcimadamore Jan 18, 2021
7757c10
Switch default for MemorySegment::allocateNative to be cleaner-enabled.
mcimadamore Jan 18, 2021
5dba87d
Simplify implementation of LibrariesHelper to use ResourceScope
mcimadamore Jan 19, 2021
1a4a8ef
Add support for non-closeable ResourceScopes
mcimadamore Jan 20, 2021
7ba07cc
First stab at javadoc
mcimadamore Jan 20, 2021
77de618
Flip defaults for short native/mapped segment factories.
mcimadamore Jan 21, 2021
63fef62
Fix NPE assertions on ResourceScope
mcimadamore Jan 21, 2021
2773f15
Fix MemoryScope javadoc
mcimadamore Jan 21, 2021
9a2e417
Tweak more defaults - now short segment factories return shared, non-…
mcimadamore Jan 21, 2021
443e166
Simplify implementation for VaList::read
mcimadamore Jan 22, 2021
16b1032
Clarify implementation of valists, by accepting a single scope.
mcimadamore Jan 22, 2021
7dc2e93
Remove ad-hoc liveness check segment from Windows VaList impl
mcimadamore Jan 22, 2021
cd89bc3
Simplify VaList API.
mcimadamore Jan 22, 2021
8b512df
Tweak benchmark after VaList changes
mcimadamore Jan 22, 2021
980544d
Add implementations for recycling/malloc/arena allocators.
mcimadamore Jan 22, 2021
1a033e8
Split ResourceScope.fork() into lock() and addOnClose().
mcimadamore Jan 22, 2021
97a60c9
Do not call cleanup action if addOnClose fails because of closed scope
mcimadamore Jan 22, 2021
475eb13
Simplify MemoryScope implementation
mcimadamore Jan 25, 2021
4e9a06f
Add extra documentation on MemoryScope
mcimadamore Jan 25, 2021
525ebbc
Fix javadoc
mcimadamore Jan 25, 2021
145b122
Remove spurious debug print statement
mcimadamore Jan 25, 2021
ae9a8e4
Minor javadoc tweaks
mcimadamore Jan 25, 2021
a0e9151
Fix javadoc of ResourceScope::close
mcimadamore Jan 25, 2021
689c94b
Another javadoc fix
mcimadamore Jan 25, 2021
fb226da
Tweak API for recycling allocator.
mcimadamore Jan 25, 2021
2847f8b
Centralize testing for all scoped entities
mcimadamore Jan 26, 2021
87d12dc
Generalize NativeAllocator to cover heap segments too (->SegmentAlloc…
mcimadamore Jan 28, 2021
2aaf78b
* Remove SharedUtils.Allocator
mcimadamore Jan 29, 2021
da936ac
Add SegmentAllocator support to CLinker
mcimadamore Jan 29, 2021
cb762d7
Enhance and fix the CallOverhead benchmark:
mcimadamore Feb 4, 2021
c96fd3d
Merge branch 'foreign-memaccess+abi' into resourceScope
mcimadamore Feb 19, 2021
bbccc3a
Fix most test failures
mcimadamore Feb 19, 2021
2fb8a8f
ALl tests pass
mcimadamore Feb 19, 2021
db0220f
Fix adaptation type comments
mcimadamore Feb 19, 2021
8e564e5
Add benchmark for strlen with recyclig scope
mcimadamore Feb 19, 2021
16cec3f
Tweak logic by which CLinker adds a SegmentAllocator argument
mcimadamore Feb 19, 2021
14d4956
Updated CLinker javadoc
mcimadamore Feb 19, 2021
2b584e4
Rename allocator factories, and add a factory which takes a scope.
mcimadamore Feb 19, 2021
9d60bd9
Fix javadoc in SegmentAllocator
mcimadamore Feb 19, 2021
35968a7
Remove linker implementation dependencies on NativeScope
mcimadamore Feb 19, 2021
0c37bf3
Fix javadoc issue in CLinker
mcimadamore Feb 19, 2021
3af1f54
Remove NativeScope
mcimadamore Feb 24, 2021
8b2894b
Clarify javadoc of ResourceScope::Lock
mcimadamore Feb 24, 2021
941edbc
Simplify arena allocators
mcimadamore Feb 25, 2021
04de173
Add useful overloads in CLinker:
mcimadamore Feb 26, 2021
a0a9dc7
Fix benchmark for implicit deallocation
mcimadamore Feb 26, 2021
9fa07a7
Simplify ResourceScope API by removing closeable bit.
mcimadamore Feb 26, 2021
67e1831
Merge branch 'foreign-memaccess+abi' into resourceScope
mcimadamore Mar 1, 2021
3f7b263
Fix some issues in upcall handler (Jorn)
mcimadamore Mar 1, 2021
ceb0774
Fix Windows VaList to add eager scope check on creation/copy
mcimadamore Mar 1, 2021
91e2794
Fix issues with GC cleaned upcall
mcimadamore Mar 1, 2021
167e25f
Fix TestResourceScope
mcimadamore Mar 1, 2021
fec4b65
Add overload for restricted VaList factory which takes a scope
mcimadamore Mar 1, 2021
321b389
Remove comments from TestDowncall
mcimadamore Mar 2, 2021
91ed099
Add assertion when functions are void
mcimadamore Mar 2, 2021
16f983a
Slightly clarify ProgrammableInvoker::specialize
mcimadamore Mar 2, 2021
092eca2
Fix javadoc
mcimadamore Mar 3, 2021
3e1399b
Rename ResourceScope::Lock
mcimadamore Mar 4, 2021
bf0ee80
Fix whitespaces
mcimadamore Mar 4, 2021
d04ee60
Fix issue in ResourceScope::ofShared() javadoc
mcimadamore Mar 5, 2021
5db5e3b
Address javadoc issues in review comments
mcimadamore Mar 10, 2021
df7eb2b
Address more review comments on MemoryScope/ResourceList
mcimadamore Mar 11, 2021
472b214
Address more review comments (VaList/SharedUtils/tests)
mcimadamore Mar 11, 2021
9d1ba85
Revert changes to SharedUtils::adaptDowncall
mcimadamore Mar 11, 2021
6328866
Cleanup SharedUtils::wrapWithAllocator and Binding.Context
mcimadamore Mar 11, 2021
69c084f
Improve arena allocator implementation:
mcimadamore Mar 11, 2021
924f8e5
Refactor MemoryScope and ResourceList hierarchies
mcimadamore Mar 11, 2021
465df11
Clarify javadoc for SegmentAllocator::arenaUnbounded
mcimadamore Mar 11, 2021
5710a35
Remove spurious println
mcimadamore Mar 11, 2021
d2c6b68
Update src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/…
mcimadamore Mar 12, 2021
65e5d63
Update src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/…
mcimadamore Mar 12, 2021
8335568
Update src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/A…
mcimadamore Mar 12, 2021
f3741fc
Update src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/…
mcimadamore Mar 12, 2021
81c4d9b
Update src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/C…
mcimadamore Mar 12, 2021
b5bdc61
Update src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/C…
mcimadamore Mar 12, 2021
285644a
Update src/jdk.incubator.foreign/share/classes/jdk/internal/foreign/C…
mcimadamore Mar 12, 2021
9c5af18
Address review comments
mcimadamore Mar 12, 2021
c19b04a
Update src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/…
mcimadamore Mar 12, 2021
5592426
Merge branch 'resourceScope' of https://github.com/mcimadamore/panama…
mcimadamore Mar 12, 2021
8545160
Address rest of review comments from Paul
mcimadamore Mar 12, 2021
195ba0f
Fix javadoc of SegmentAllocator
mcimadamore Mar 12, 2021
d02e86f
Add ResourceScope overloads in CLinker
mcimadamore Mar 18, 2021
0ea68e1
Update src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/…
mcimadamore Mar 19, 2021
697d179
Update src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/…
mcimadamore Mar 19, 2021
15431e7
Update src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/…
mcimadamore Mar 19, 2021
79dde96
Update src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/…
mcimadamore Mar 19, 2021
5de033e
* More javadoc fixes
mcimadamore Mar 19, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -628,7 +628,7 @@ interface VaList extends Addressable {

/**
* Constructs a new {@code VaList} instance out of a memory address pointing to an existing C {@code va_list},
* backed by the global resource scope (see {@link ResourceScope#globalScope()}).
* backed by the {@link ResourceScope#globalScope() global} resource scope.
* <p>
* This method is <em>restricted</em>. Restricted method are unsafe, and, if used incorrectly, their use might crash
* the JVM or, worse, silently result in memory corruption. Thus, clients should refrain from depending on
Original file line number Diff line number Diff line change
@@ -92,7 +92,7 @@ default MemoryAddress address() {
* Returns a shared native memory segment with given size, and whose base address is this address. This method
* can be useful when interacting with custom native memory sources (e.g. custom allocators), where an address to some
* underlying memory region is typically obtained from native code (often as a plain {@code long} value).
* The returned segment is associated with the global resource scope (see {@link ResourceScope#globalScope()}).
* The returned segment is associated with the {@link ResourceScope#globalScope() global} resource scope.
* <p>
* Clients should ensure that the address and bounds refers to a valid region of memory that is accessible for reading and,
* if appropriate, writing; an attempt to access an invalid memory location from Java code will either return an arbitrary value,

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -78,9 +78,9 @@
* associated with this resource scope. Any attempt to perform resource access from a thread other than the
* owner thread will result in a runtime failure.
* <p>
* Shared resource scopes (see {@link #ofShared()}), support strong thread-confinement guarantees. A shared resource scope
* has no owner thread; as such resources associated with this scope can be accessed by multiple threads. This might be useful
* when multiple threads need to access the same resource concurrently (e.g. in the case of parallel processing). For instance, a client
* Shared resource scopes (see {@link #ofShared()}), on the other hand, have no owner thread; as such resources associated
* with this shared resource scopes can be accessed by multiple threads. This might be useful when multiple threads need
* to access the same resource concurrently (e.g. in the case of parallel processing). For instance, a client
* might obtain a {@link Spliterator} from a shared segment, which can then be used to slice the segment and allow multiple
* threads to work in parallel on disjoint segment slices. The following code can be used to sum all int values in a memory segment in parallel:
*
Original file line number Diff line number Diff line change
@@ -461,7 +461,7 @@ static SegmentAllocator arenaUnbounded(ResourceScope scope) {
}

/**
* Returns a native allocator which responds to allocation requests by recycling a single segment; that is,
* Returns a segment allocator which responds to allocation requests by recycling a single segment; that is,
* each new allocation request will return a new slice starting at the segment offset {@code 0} (alignment
* constraints are ignored by this allocator). This can be useful to limit allocation requests in case a client
* knows that they have fully processed the contents of the allocated segment before the subsequent allocation request
Original file line number Diff line number Diff line change
@@ -338,11 +338,11 @@ private boolean isSet(int mask) {

private int checkArraySize(String typeName, int elemSize) {
if (length % elemSize != 0) {
throw new UnsupportedOperationException(String.format("Segment size is not a multiple of %d. Size: %d", elemSize, length));
throw new IllegalStateException(String.format("Segment size is not a multiple of %d. Size: %d", elemSize, length));
}
long arraySize = length / elemSize;
if (arraySize > (Integer.MAX_VALUE - 8)) { //conservative check
throw new UnsupportedOperationException(String.format("Segment is too large to wrap as %s. Size: %d", typeName, length));
throw new IllegalStateException(String.format("Segment is too large to wrap as %s. Size: %d", typeName, length));
}
return (int)arraySize;
}
Original file line number Diff line number Diff line change
@@ -31,6 +31,7 @@ MemorySegment newSegment(long size, long align) {

@Override
public MemorySegment allocate(long bytesSize, long bytesAlignment) {
checkConfinementIfNeeded();
if (Utils.alignUp(bytesSize, bytesAlignment) > MAX_ALLOC_SIZE) {
return newSegment(bytesSize, bytesAlignment);
}
@@ -62,6 +63,13 @@ private MemorySegment trySlice(long bytesSize, long bytesAlignment) {
}
}

private void checkConfinementIfNeeded() {
Thread segmentThread = segment.scope().ownerThread();
if (segmentThread != null && segmentThread != Thread.currentThread()) {
throw new IllegalStateException("Attempt to allocate outside confinement thread");
}
}

public static class BoundedArenaAllocator extends ArenaAllocator {

public BoundedArenaAllocator(ResourceScope scope, long size) {
10 changes: 5 additions & 5 deletions test/jdk/java/foreign/TestArrays.java
Original file line number Diff line number Diff line change
@@ -108,7 +108,7 @@ public void testArrays(Consumer<MemorySegment> init, Consumer<MemorySegment> che
}

@Test(dataProvider = "elemLayouts",
expectedExceptions = UnsupportedOperationException.class)
expectedExceptions = IllegalStateException.class)
public void testTooBigForArray(MemoryLayout layout, Function<MemorySegment, Object> arrayFactory) {
MemoryLayout seq = MemoryLayout.ofSequence((Integer.MAX_VALUE * layout.byteSize()) + 1, layout);
//do not really allocate here, as it's way too much memory
@@ -117,19 +117,19 @@ public void testTooBigForArray(MemoryLayout layout, Function<MemorySegment, Obje
}

@Test(dataProvider = "elemLayouts",
expectedExceptions = UnsupportedOperationException.class)
expectedExceptions = IllegalStateException.class)
public void testBadSize(MemoryLayout layout, Function<MemorySegment, Object> arrayFactory) {
if (layout.byteSize() == 1) throw new UnsupportedOperationException(); //make it fail
if (layout.byteSize() == 1) throw new IllegalStateException(); //make it fail
try (ResourceScope scope = ResourceScope.ofConfined()) {
MemorySegment segment = MemorySegment.allocateNative(layout.byteSize() + 1, layout.byteSize(), scope);
arrayFactory.apply(segment);
}
}

@Test(dataProvider = "elemLayouts",
expectedExceptions = UnsupportedOperationException.class)
expectedExceptions = IllegalStateException.class)
public void testArrayFromClosedSegment(MemoryLayout layout, Function<MemorySegment, Object> arrayFactory) {
MemorySegment segment = MemorySegment.allocateNative(layout);
MemorySegment segment = MemorySegment.allocateNative(layout, ResourceScope.ofConfined());
segment.scope().close();
arrayFactory.apply(segment);
}
2 changes: 1 addition & 1 deletion test/jdk/java/foreign/TestByteBuffer.java
Original file line number Diff line number Diff line change
@@ -454,7 +454,7 @@ public void testBufferOnClosedScope() {
byteBuffer.get(); // should throw
}

@Test(expectedExceptions = UnsupportedOperationException.class)
@Test(expectedExceptions = IllegalStateException.class)
public void testTooBigForByteBuffer() {
MemorySegment segment = MemoryAddress.NULL.asSegmentRestricted(Integer.MAX_VALUE + 10L);
segment.asByteBuffer();