Adds support for writing macros in IonManagedWriter_1_1#934
Conversation
bfba2e3 to
770c81e
Compare
| public void writeBlob(byte[] value, int start, int len) | ||
| throws IOException; | ||
|
|
||
| public default void writeObject(WriteAsIon obj) { |
There was a problem hiding this comment.
🗺️ I'm not married to this name. Suggestions for a better name are welcome.
| /** | ||
| * Writes this object to an IonWriter capable of producing macro invocations. | ||
| */ | ||
| fun writeToMacroAware(writer: MacroAwareIonWriter) = writeTo(writer as IonWriter) |
There was a problem hiding this comment.
🗺️ I didn't want to overload writeTo. Naming suggestions welcome here if you can think of a better one.
| if (closed) return | ||
| confirm(depth() == 0) { "Cannot call finish() while in a container" } | ||
| confirm(numAnnotations == 0) { "Cannot call finish with dangling annotations" } | ||
| output.flush() |
There was a problem hiding this comment.
🗺️ This was the bug in the finish() (now flush()) implementation.
| * TODO: What package does this really belong in? | ||
| * See also [ManagedWriterOptions], [SymbolInliningStrategy], and [DelimitedContainerStrategy]. | ||
| * | ||
| * See also [ManagedWriterOptions_1_1], [SymbolInliningStrategy], and [LengthPrefixStrategy]. |
There was a problem hiding this comment.
🗺️ Moved so that the KDoc links will work. For some reason, KDoc links didn't seem to work when the IDE sees it's in a "TODO" item.
| // TODO: See if there's any benefit to using a smaller number type, if we can | ||
| // memoize this in the macro definition, or replace it with a list of precomputed | ||
| // step-out indices. | ||
| /** Tracks where and how many times to step out. */ | ||
| val numberOfTimesToStepOut = IntArray(macro.body.size + 1) |
There was a problem hiding this comment.
🗺️ It just occurred to me that precomputing and/or memoizing this information in the TemplateMacro would also make it easier to break up this large method into smaller methods. The reason I didn't break this up into smaller methods is because of the awkwardness of passing around the numberOfTimesToStepOut array.
I'll leave that as a TODO for later though.
| EEXP -> presenceBitmapStack.peek().signature[currentContainer.numChildren].type.taglessEncodingKind | ||
| EEXP -> { | ||
| val signature = presenceBitmapStack.peek().signature | ||
| if (currentContainer.numChildren >= signature.size) throw IllegalArgumentException("Too many arguments for macro with signature $signature") | ||
| signature[currentContainer.numChildren].type.taglessEncodingKind | ||
| } |
There was a problem hiding this comment.
🗺️ Previously, this would just throw an IndexOutOfBoundsException (or something like that). I ran into this unhelpful exception while I was testing things, so I put in a more useful message.
| val NEVER_INLINE = SymbolInliningStrategy { _, _ -> false } | ||
| val NEVER_INLINE = object : SymbolInliningStrategy { | ||
| override fun shouldWriteInline(symbolKind: SymbolKind, text: String): Boolean = false | ||
| override fun toString(): String = "NEVER_INLINE" | ||
| } |
There was a problem hiding this comment.
🗺️ The only real difference here is that instead of a lambda expression, we have an anonymous class with a useful toString() implementation.
| companion object { | ||
| // This is a very long macro name, but by using the qualified class name, | ||
| // there is almost no risk of having a name conflict with another macro. | ||
| private val MACRO_NAME = Point2D::class.qualifiedName!!.replace(".", "_") |
There was a problem hiding this comment.
🗺️ It would be even safer to replace . with $, but that was not playing nicely with the expected output in the test case because Kotlin uses $ to indicate string interpolation.
| templateBody { | ||
| struct { | ||
| fieldName("x") | ||
| variable(0) | ||
| fieldName("y") | ||
| variable(1) | ||
| } | ||
| } |
There was a problem hiding this comment.
🗺️ If we want people to be able to define macros for their own types, I think we do need a builder of some sort. It's so much easier to use a builder than it is to construct the expressions directly.
There was a problem hiding this comment.
🗺️ I rearranged this file so that the Ion 1.0 symbols are first, and any added Ion 1.1 symbols come afterwards. I also added ANNOTATE, EXPORT, TDL_EXPRESSION_GROUP, LITERAL, and MAKE_SEXP.
There was a problem hiding this comment.
🗺️ There is a lot of repetitive code here. Most of it is interfaces that define what methods may be called when stepped into various container types for each of DataModelExpression, TemplateBodyExpression, and EExpressionBodyExpression. This is what lets us elegantly write things like
templateBody {
list {
int(1)
variable(0)
macro("foo") {
string("bar")
}
}
}
zslayton
left a comment
There was a problem hiding this comment.
LGTM! Coupla small things to consider before merging.
| // Check the current macro table | ||
| var id = macroTable[macro] | ||
| if (id != null) return id | ||
| // Check the to-be-appended macros | ||
| id = newMacros[macro] | ||
| if (id != null) return id | ||
| // Add to the to-be-appended symbols |
There was a problem hiding this comment.
Should this check the newMacros first so new definitions shadow the old ones?
There was a problem hiding this comment.
No—at least not here—because this uses deep equality of the macro signature and body. (In practice, though, it should be short-circuited by a this === other check most of the time.)
However, your question is making me rethink the addMacro(...): MacroRef function because MacroRef could outlive an encoding context with no good way to determine whether it's still valid.
| repeat(numberOfTimesToStepOut[index]) { stepOut() } | ||
| } | ||
|
|
||
| when (expression) { |
There was a problem hiding this comment.
Should the writing logic live in the WriteAsIon impl for the various expression kinds?
| newSymbols.forEach { (text, _) -> writeString(text) } | ||
| stepOut() | ||
| stepOut() | ||
| private fun resetEncodingContext() { |
There was a problem hiding this comment.
After this is called, how does the writer know that it needs to write a new encoding directive before the next value?
There was a problem hiding this comment.
The next time the user values are flushed, if newSymbols or newMacros is non-empty, then the managed writer will write an encoding directive before it flushes the user values. If both are empty, then we know that there are no SIDs or E-Expressions in the user values being flushed, and so no encoding directive is needed because it makes no difference.
Now that I think about it, there might be an edge case for Ion text if a system macro was being shadowed—it would end up writing incorrect data. This can be solved by making the managed writer emit an IVM if a value is written after finish() has been called.
| } | ||
|
|
||
| @Test | ||
| fun `write an e-expression by name 2`() { |
There was a problem hiding this comment.
Is this just a duplicate of the previous test?
There was a problem hiding this comment.
+1, I think this will be very important for readability.
770c81e to
a84b4ed
Compare

Issue #, if available:
#733, #737, #742
Description of changes:
I've split it into multiple commits to make it easier to review.
finish()because it didn't actually flush the output.IonWriterwith a default implementation.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.